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Commonality,  an  increasingly  popular  strategy  in  developing  complex 
defense  projects,  leverages  sharing  or  reuse  across  projects  to  signifi¬ 
cantly  reduce  life-cycle  costs.  Despite  its  potential  within  DoD  as  a  best 
practice,  programs  focused  on  commonality  have  met  with  mixed  success. 
This  article  argues  that  commonality  strategies  must  be  matched  with 
complementary  acquisition  strategies  to  improve  outcomes.  Full,  open 
competition  is  not  the  best  acquisition  strategy  if  commonality  can  unlock 
life-cycle  affordability.  Metrics  and  payment  structures  must  consider 
the  commonality  goals  to  be  achieved;  otherwise,  contractor  motivations 
and  government  goals  will  be  misaligned.  The  recommendations  in  this 
article  draw  on  commonality  research  conducted  on  behalf  of  the  National 
Aeronautics  and  Space  Administration  (NASA),  which  examined  19  DoD, 
commercial,  and  NASA  case  studies. 
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With  the  fast-paced  nature  of  technology,  rapidly  fielding  systems  has  never 
been  more  important.  Success  depends  on  well-defined  requirements  and 
the  ability  to  rapidly  respond  to  change  during  and  after  deployment.  The 
inability  to  rapidly  respond  may  cause  the  system  to  become  obsolete  before 
initial  fielding.  Creating  a  structure  where  processes  allow  for  changes 
during  system  development  requires  restructuring  system  development 
values  and  principles  at  all  levels.  This  article  addresses  progress  toward 
agility  and  defines  agile  values  and  principles  being  used  by  agile  orga¬ 
nizations  in  the  Business,  System,  and  Software  Aspects.  It  also  defines 
operationally  effective  agile  practices  being  utilized  to  implement  those 
values  and  principles  that  provide  a  starting  point  for  inserting  agility  into 
the  system  development  process. 
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As  the  volume  of  government  contracting  increases,  so  does  the  importance 
of  monitoring  government  contractors  to  guard  against  Organizational 
Conflict  of  Interest  (OCI).  For  contracting  officers  to  identify  OCIs,  they 
must  be  able  to  identify  the  relevant  business  interests  of  a  contractor's 
affiliates.  This  information  may  be  private  or  not  easily  obtained.  Using 
newly  released  data  to  develop  preliminary  visualizations  of  contractor 
organizational  structures  shows  the  organizational  structure  of  many 
contractors  to  be  complex  and  multinational.  The  complexity  and  the  lack 
of  easily  available  public  information  make  it  very  unlikely  that  contracting 
officers  could  identify  OCIs  without  substantial  improvements  in  govern¬ 
ment  data  collection. 
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Learning  curves  are  useful  for  assessing  performance  improvement  due 
to  the  positive  impact  of  learning.  In  recent  years,  the  deleterious  effects 
of forgetting  have  also  been  recognized.  Workers  experience  forgetting  or 
decline  in  performance  over  time.  Consequently,  contemporary  learning 
curves  have  attempted  to  incorporate  forgetting  components  into  learning 
curves.  An  area  of  increasing  interest  is  the  study  of  how  fast  and  how 
far  the  forgetting  impact  can  influence  overall  performance.  This  article 
introduces  the  concept  of  half-life  analysis  of  learning  curves  using  the 
concept  of  growth  and  decay,  with  particular  emphasis  on  applications 
in  the  defense  acquisition  process.  The  computational  analysis  of  the 
proposed  technique  lends  itself  to  applications  for  designing  training  and 
retraining  programs  for  the  Defense  Acquisition  Workforce. 
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As  U.S.  Government  facilities  age  and  new  facilities  are  constructed, 
the  need  to  hire  contractors  for  an  increasing  number  of  government 
construction  projects  is  imperative.  The  current  government  technical 
evaluation  for  contractor  selection  is  less  than  optimal.  This  article 
introduces  an  alternative  technical  evaluation  methodology  to  the  current 
government  contractor  selection  process:  a  Decision  Cost  Model  (DCM) 
that  can  be  applied  to  ensure  cost-efficient  contractors  are  selected  in 
awarding  construction  contracts.  Applying  the  DCM  ensures  contractors 
with  the  lowest  expected  total  cost  are  recommended  for  project  awards. 
Also  presented  are  ways  DCM  can  be  applied  to  increase  efficiency  in 
the  selection  process  for  future  government  construction  projects,  while 
simultaneously  meeting  taxpayers'  expectations  of  receiving  maximum 
value  for  their  tax  dollars. 
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This  issue's  theme,  “The  Visible  Hand  of  Defense 
Systems  Management :,”  reflects  the  fact  that  the  devel¬ 
opment  and  fielding  of  military  systems  and  services 
has  never  been  directed  solely  by  the  “invisible  hand” 
of  market  mechanisms,  but  rather  has  always  relied 
on  strong  guidance  by  the  “visible  hand”  of  deliberate,  well-considered 
management  of  the  acquisition  process.  The  origins  of  this  process  are 
engagingly  described  in  Alfred  Chandler's  The  Visible  Hand:  The  Manage¬ 
rial  Revolution  in  American  Business,  this  issue's  selection  for  the  Defense 
Acquisition  Professional  Reading  List  and  reviewed  by  Dr.  Nayantara 
Hensel,  a  member  of  the  Defense  ARJ’ s  Research  Advisory  Board. 


The  first  article,  “Relieving  Joint  Pain”  by  Anthony  Wicht  and  Edward 
Crawley,  describes  how  strategic  planning  for  commonality  among  acqui¬ 
sition  programs  can  translate  directly  into  life-cycle  savings.  The  next 
article,  “Inserting  Agility  in  System  Development”  by  Matthew  Kennedy 
and  Dan  Ward,  argues  that  a  hands-on  approach  to  change  management 
during  the  system  development  cycle  can  avoid  system  obsolescence  before 
initial  fielding.  Melissa  Thomas'  article,  “Identifying  Organizational 
Conflict  of  Interest,”  suggests  the  need  for  a  robust  means  of  collecting 
information  and  monitoring  government  contractors  to  guard  against 
organizational  conflict  of  interest  during  the  acquisition  process.  Adedeji 
Badiru,  in  “Half-Life  Learning  Curves  in  the  Defense  Acquisition  Life 
Cycle,”  introduces  the  concept  of  half-life  analysis  of  learning  curves  (i.e., 
reflecting  the  decay  of  some  learning  even  as  other  learning  increases)  as 
a  potential  tool  for  planning  career  and  training  strategies  in  the  defense 
acquisition  process.  Finally,  Victor  Apodaca,  in  the  online-only  article 
“Decision  Cost  Model  for  Contractor  Selection,”  introduces  an  alternative 
technical  evaluation  methodology  to  the  current  government  contractor 
selection  process  using  a  statistical  model  to  identify  the  contractor  with 
the  lowest  expected  total  cost. 
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The  Defense  Acquisition  Research  Agenda  is  intended  to  make 
researchers  aware  of  the  topics  that  are,  or  should  be,  of  particular 
concern  to  the  broader  defense  acquisition  community  throughout 
the  government,  academic,  and  industrial  sectors.  The  purpose  of 
conducting  research  in  these  areas  is  to  provide  solid,  empirically  based 
findings  to  create  a  broad  body  of  knowledge  that  can  inform  the  devel¬ 
opment  of  policies,  procedures,  and  processes  in  defense  acquisition, 
and  to  help  shape  the  thought  leadership  for  the  acquisition  community. 


Each  issue  of  the  Defense  ARJ  will  include  a  different  selection  of 
research  topics  from  the  overall  agenda,  which  is  at: 
http://www.dau.mil/research/Pages/researchareas.aspx 


Measuring  the  effects  of  competition 

•  What  means  are  there  (or  can  be  developed)  to  measure  the  effect 
on  defense  acquisition  costs  of  maintaining  an  industrial  base  in 
various  sectors? 


•  What  means  exist  (or  can  be  developed)  of  measuring  the  effect  of 
utilizing  defense  industrial  infrastructure  for  commercial  manu¬ 
facture  in  growth  industries?  In  other  words,  can  we  measure  the 
effect  of  using  defense  manufacturing  to  expand  the  buyer  base? 

•  What  means  exist  (or  can  be  developed)  to  determine  the  degree  of 
openness  that  exists  in  competitive  awards? 


•  What  are  the  different  effects  of  the  two  best  value  source  selec¬ 
tion  processes  (tradeoff  vs.  lowest  price  technically  acceptable)  on 
program  cost,  schedule,  and  performance? 


Strategic  competition 

•  Is  there  evidence  that  competition  between  system  portfolios  is  an 
effective  means  of  controlling  price  and  costs? 

•  Does  lack  of  competition  automatically  mean  higher  prices?  For 
example,  is  there  evidence  that  sole  source  can  result  in  lower 
overall  administrative  costs  at  both  the  government  and  industry 
levels,  to  the  effect  of  lowering  total  costs? 

•  What  are  the  long-term  historical  trends  for  competition  guidance 
and  practice  in  defense  acquisition  policies  and  practices? 

•  To  what  extent  are  contracts  being  awarded  non-competitively  by 
congressional  mandate,  for  policy  interest  reasons?  What  is  the 
effect  on  contract  price  and  performance? 

•  What  means  are  there  (or  can  be  developed)  to  determine  the  degree 
to  which  competitive  program  costs  are  negatively  affected  by 
laws  and  regulations  such  as  the  Berry  Amendment,  Buy-America 
Acts,  etc.? 
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Relieving  Joint  Pain: 

Planning  Government 
Acquisition  of  Complex 
Common  Systems 


]  Anthony  C.  Wicht  and  Edward  F.  Crawley 


Commonality,  an  increasingly  popular  strategy  in  devel¬ 
oping  complex  defense  projects,  leverages  sharing  or 
reuse  across  projects  to  significantly  reduce  life-cycle 
costs.  Despite  its  potential  within  DoD  as  a  best  practice, 
programs  focused  on  commonality  have  met  with  mixed 
success.  This  article  argues  that  commonality  strategies 
must  be  matched  with  complementary  acquisition  strate¬ 
gies  to  improve  outcomes.  Full,  open  competition  is  not 
the  best  acquisition  strategy  if  commonality  can  unlock 
life-cycle  affordability.  Metrics  and  payment  structures 
must  consider  the  commonality  goals  to  be  achieved; 
otherwise,  contractor  motivations  and  government  goals 
will  be  misaligned.  The  recommendations  in  this  article 
draw  on  commonality  research  conducted  on  behalf 
of  the  National  Aeronautics  and  Space  Administration 
(NASA),  which  examined  19  DoD,  commercial,  and  NASA 
case  studies. 
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Commonality,  the  sharing  of  parts  or  processes  across  different 
products,  has  long  been  popular  in  commercial  industries  such  as  auto¬ 
motives  and  electronics  because  it  reduces  life-cycle  costs  and  improves 
reliability.  Today,  commonality  is  enjoying  increasing  interest  from  the 
defense  industry  as  the  emphasis  on  life-cycle  affordability  strengthens 
(Brown  &  Flowe,  2005).  Joint  Services  programs  such  as  the  Joint  Strike 
Fighter  (JSF)  and  Joint  Tactical  Radio  System  (JTRS)  develop  partially 
common  systems  that  meet  the  needs  of  different  Services.  Other  pro¬ 
grams  such  as  the  National  Polar-orbiting  Operational  Environmental 
Satellite  System  (NPOESS)  attempt  to  exploit  commonality  between 
the  needs  of  DoD  and  other  agencies.  Even  within  a  single  Service,  com¬ 
monality  is  a  useful  strategy,  for  example  in  the  adaptation  of  the  M577 
command  post  vehicle  from  the  M113  armored  personnel  carrier,  which 
simplifies  development  and  decreases  logistics  costs  (Terry,  Jackson, 
Ryley,  Jones,  &  Wormell,  1991).  The  major  benefit  from  commonality 
is  affordability,  which  has  increased  attention  on  the  strategy  in  recent 
years  as  defense  budgets  have  tightened.  Commonality  will  continue  to 
be  an  important  tool  for  acquisition  professionals  while  cost  pressure  on 
defense  budgets  remains  high. 

Despite  commonality's  promise  of  increased  affordability,  the  perfor¬ 
mance  of  defense  acquisition  based  on  commonality  has  lagged  behind 
comparable  commercial  projects.  NPOESS  was  canceled,  JTRS  was 
fundamentally  restructured  in  2005,  and  the  threat  of  Nunn-McCurdy 
breaches  is  still  a  factor  in  the  program  stability  of  the  JSF.  We  hypoth¬ 
esized  that  the  different  acquisition  environments  between  commercial 
and  defense  commonality  projects  were  partly  responsible.  Therefore, 
the  objective  of  the  research  was  to  examine  current  government  acquisi¬ 
tion  practices  in  commonality,  and  synthesize  a  best-practice  acquisition 
strategy  for  future  commonality  projects. 

Extensive  literature  on  commonality  already  exists;  however,  it 
focuses  on  the  application  of  commonality  platforms  to  business  strategy 
(Meyer  &  Lehnerd,  1997;  Robertson  &  Ulrich,  1998)  or  stops  at  the  identi¬ 
fication  of  technically  feasible  commonality.  (For  examples  from  the  DoD 
context,  see  the  RAND  report  by  Held,  Newsome,  &  Lewis,  2008.)  Boas 
and  Rhodes  developed  management  approaches  to  commonality  (Boas  & 
Crawley,  2006;  Rhodes,  2010),  which  built  on  the  more  general  advice  in 
the  platforming  literature,  but  no  work  on  commonality  was  found  that 
specifically  examined  acquisition.  In  the  acquisition  literature,  Scherer's 
unsurpassed  economic  analysis  of  the  effect  of  acquisition  strategy  on 
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weapon  effectiveness  in  the  DoD  informed  much  of  the  analysis  in  the 
second  half  of  this  study  (Scherer,  1964).  Additionally,  handbooks  for 
the  acquisition  professional  emphasize  that  the  acquisition  approach 
must  take  into  account  particulars  of  the  acquisition  at  hand  (Defense 
Acquisition  University,  2011;  Rendon  &  Snider,  2008).  However,  no  piece 
of  the  acquisition  literature  delivered  specific  advice  for  acquiring  com¬ 
mon  systems. 


Objective  and  Outline 

To  fill  this  gap,  this  article  aims  to  answer  four  questions: 

•  Which  principles  from  the  extensive  literature  on  commer¬ 
cial  commonality  form  necessary  background  knowledge 
for  the  defense  acquisition  professional? 

•  Which  acquisition  approaches  represent  best  practice 
when  formulating  an  acquisition  strategy  for  a  new  Joint 
Services  program,  or  an  intra-Service  program  involving 
commonality? 

•  Which  additional  contract  terms  such  as  payment  pro¬ 
visions  or  intellectual  property  considerations  improve 
acquisition  outcomes  in  the  commonality  environment? 

•  Are  the  acquisition  regulations  flexible  enough  to  permit 
best-practice  commonality  acquisition? 

At  the  outset,  it  is  important  to  note  that  commonality  is  not  the 
only  approach  for  improving  acquisition  outcomes.  Other  product  devel¬ 
opment  philosophies  such  as  flexibility,  robustness,  interoperability, 
adaptability,  and  open  architectures  are  widely  discussed  in  the  lit¬ 
erature  and  can  improve  the  performance  of  acquisitions.  Contrasting 
these  alternative  approaches  is  beyond  the  scope  of  this  article;  however, 
similar  analysis  to  that  presented  in  this  article  could  be  used  to  craft 
acquisition  strategies  that  complement  these  development  philosophies. 

This  article  will  first  summarize  the  research  method  and  sketch  the 
case  studies,  followed  by  a  presentation  of  the  background  concepts  on 
commonality,  which  distinguish  commonality-focused  acquisitions  from 
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single-product  acquisitions.  Finally,  the  article  will  turn  specifically 
to  acquisition  approaches,  analyzing  the  alternative  ways  in  which  an 
acquisition  could  be  approached  and  recommending  specific  strategies. 

Research  Method 

The  research  in  this  article  builds  on  19  commonality  case  studies 
conducted  by  the  MIT  Space  Systems  Architecture  Group  (Boas,  2008; 
Hofstetter,  2009;  Rhodes,  2010;  Cameron,  2011;  Wicht,  2011).  Of  these,  16 
cases  informed  the  general  commonality  principles  presented  in  the  first 
half  of  this  article,  and  were  instrumental  in  developing  the  concepts 
and  process  maps  that  identify  commonality  as  a  best  practice.  A  further 
three  case  studies  conducted  by  the  authors  were  specifically  targeted 
at  acquisition  and  were  complemented  by  17  short  interviews  with  DoD 
and  NASA  personnel  involved  in  the  acquisition  process.  The  follow¬ 
ing  paragraphs  briefly  describe  the  acquisition  case  studies;  however, 
full  reports  on  the  cases  are  available  in  Wicht  (2011).  The  three  cases 
examined  in  detail  were  JTRS  and  two  nongovernment  launch  vehicle 
manufacturers  who  requested  anonymity. 

JTRS  was  a  Joint  Services  project  to  produce  software-defined 
radios  that  were  interoperable  among  the  Services.  Commonality  of 
software  between  the  radios  was  intended  to  deliver  development  and 
maintenance  savings  as  well  as  performance  benefits  from  improved 
interoperability.  The  initial  architecture  for  the  radios  was  designed 
by  a  working  group  including  government  and  industry  representa¬ 
tives.  The  detailed  design  and  manufacture  of  the  radio  hardware  and 
software  were  distributed  across  multiple  contractors.  Interviews  with 
a  range  of  former  DoD  personnel  and  consultants  involved  with  JTRS 
were  undertaken  to  capture  how  the  acquisition  approach  affected  the 
realized  commonality. 

The  two  commercial  launch  vehicle  manufacturers  both  produce 
families  of  launch  vehicles  for  DoD,  NASA,  and  commercial  applications. 
Both  have  worked  as  contractors  to  DoD  or  NASA  previously.  The  launch 
vehicle  manufacturers  were  examined  because  each  had  development 
tasks  comparable  in  complexity  to  those  undertaken  by  government 
agencies  like  DoD  and  NASA. 
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Two  avenues  of  investigation  were  pursued.  First,  in  their  position 
as  system  integrator,  how  did  the  commercial  companies  structure  their 
acquisitions  to  develop  and  maintain  the  right  level  of  commonality? 
Second,  in  their  position  as  a  contractor  to  DoD  or  NASA,  what  potential 
pitfalls  did  they  see  in  the  commonality  acquisition  strategies  proposed? 

After  conducting  the  case  studies,  acquisition  approaches  uncov¬ 
ered  during  the  short  interviews  and  case  studies  were  qualitatively 
evaluated  against  the  commonality  process  map.  Acquisition  structures 
were  then  graded  as  Good,  Moderate,  or  Poor  according  to  how  well 
the  structure  itself  facilitated  the  processes  that  underpin  common¬ 
ality.  A  second  pass  through  the  Good  and  Moderate  approaches  was 
then  undertaken  to  refine  contractor  payments,  incentives,  intellectual 
property,  and  other  provisions  to  develop  acquisition  strategies  that  best 
implement  commonality. 


Definitions 

To  understand  the  interaction  between  commonality  and  acquisition 
strategy,  key  commonality  definitions  and  background  are  presented 
here.  Although  no  widely  agreed-upon  definitions  for  commonality  are 
prevalent  throughout  the  defense  acquisition  community,  an  excel¬ 
lent  RAND  paper  contains  a  DoD  lexicon  for  commonality  (Newsome, 
Lewis,  &  Held,  2007).  The  following  simple  definitions  will  be  used  in 
this  article: 


Common 

Unique 

Similar 

Family 

Variant 


having  identical  elements 

the  antithesis  of  common 

some  identical  and  some  unique  elements 

a  set  of  similar  end-items  that  perform  different  functions 

any  one  member  of  a  family 


Commonality  Concepts 

Using  these  definitions,  it  is  possible  to  summarize  the  concepts 
distilled  from  the  19  case  studies  and  the  product  literature,  which  shape 
the  application  of  commonality  to  real-world  projects  that  an  acquisition 
strategy  must  address. 
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Concept  1:  Commonality  is  not  an  end  in  itself.  The  first 
concept  is  that  commonality  is  not  an  end  in  itself.  Commonality  carries 
both  advantages  and  disadvantages,  and  therefore  the  best  project  is  not 
necessarily  the  one  with  more  commonality.  The  optimum  amount  of 
commonality  is  that  amount  which  best  meets  the  customer's  needs  in 
terms  of  life-cycle  affordability  and  performance. 

Seeing  commonality  as  an  enabler  rather  than  an  end  goal  car¬ 
ries  two  implications  for  acquisition.  First,  contractors  should  not  be 
incentivized  to  target  maximum  commonality  or  fixed  percentages  of 
commonality  because  this  misaligns  contractor  incentives  and  cus¬ 
tomer  needs.  Second,  the  benefits  of  commonality  should  be  balanced 
against  the  costs  of  achieving  it.  The  acquisition  strategies  recom¬ 
mended  for  commonality  are  likely  to  cost  more  to  implement  than  full 
and  open  competition. 

Concept  2:  Realized  commonality  is  always  less 
than  initially  planned  commonality  (“divergence”)-  Boas 

demonstrated  that  the  level  of  realized  commonality  is  always  less  than 
the  level  initially  planned.  This  decrease  is  called  divergence.  From 
Concept  1,  it  follows  that  divergence  may  be  positive  or  negative  for  the 
project.  Divergence  is  positive  if  it  occurs  to  accommodate  the  emergence 
of  new  technologies,  learning  from  the  development  of  earlier  variants  or 
changes  in  the  field  conditions  for  the  product.  Divergence  is  negative  if 
it  stems  from  mismanagement  or  attempts  to  improve  the  performance 
of  individual  variants  at  the  expense  of  the  family. 

The  implication  for  acquisition  is  that  the  acquisition  structures 
must  have  controls  to  limit  detrimental  divergence,  but  not  penalize  ben¬ 
eficial  divergence.  More  generally,  foreknowledge  of  the  inevitability  of 
divergence  can  help  manage  expectations,  prepare  more  accurate  project 
budgets  and  schedules,  and  avoid  overreaction  to  a  normal  corollary  of 
commonality  development. 

Concept  3:  Commonality  projects  are  offset  in  time.  Boas 
also  showed  that  complex  development  projects  experience  time  offsets 
between  the  development  of  variants  to  lower  the  peaks  in  labor  and 
capital  demand.  Offsets  often  mean  that  the  first-in-time  variant  team 
develops  the  common  systems,  with  a  resultant  bias  toward  the  better 
defined  needs  of  the  first-in-time  variant. 
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The  implication  of  offsets  for  acquisition  are  twofold.  One,  the 
first-in-time  project  must  be  incentivized  to  consider  the  needs  of  all 
later-in-time  projects  during  the  first  development  phase.  Two,  the 
requirements  for  subsequent  variants  must  be  well  defined  when  the  first 
variant  is  undergoing  concept  studies.  This  requires  earlier  funding  for 
the  subsequent  variants. 

Concept  4:  Commonality  requires  up-front  cost  and 
delivers  benefits  later  in  the  product  life  cycle.  Offsets  lead  to 
a  consistent  cost  structure  for  commonality  projects.  The  first-in-time 
variant  bears  the  burden  of  developing  all  common  systems  before  it  is 
operational.  This  means  the  first-in-time  variant  incurs  a  cost  penalty 
relative  to  the  development  cost  of  the  other  variants.  In  a  sensible 
commonality  program,  the  additional  cost  of  commonality  is  recovered 
over  the  life  cycle  at  the  family  level  through  lower  development  costs 
for  subsequent  variants  and  more  effective  sharing  of  recurring  costs 
across  the  family. 

The  implication  for  acquisition  is  that  development  decisions  must 
be  taken  based  on  life-cycle  costs,  not  development  costs,  and  based  on 
family-level  cost-benefit  analysis,  not  variant-level  cost-benefit  analysis. 
If  the  first  variant  is  to  implement  commonality,  it  must  receive  extra 
funding  and  high-level  management  support  to  permit  spending  on  the 
up-front  costs.  Without  these  measures,  first-in-time  variants  have  no 
incentive  to  implement  commonality. 

Concept  5:  Three  commonality  strategies  exist— Reactive 
Reuse,  Building  Block,  and  Widespread  Forward.  Three 
general  commonality  strategies  are  observed  (Boas,  2008).  The  simplest, 
Reactive  Reuse,  examines  previous  products  for  elements  that  could 
be  used  again  in  the  next  variant.  The  original  products  never  planned 
for  reuse,  thus  avoiding  the  up-front  commonality  costs  pointed  out 
in  Concept  4.  However,  the  Reactive  Reuse  benefits  the  project  under 
development  because  it  reduces  development  cost  and  risk,  making  it 
an  attractive  strategy.  With  such  ad  hoc  reuse,  substantial  affordability 
improvements  are  difficult  to  achieve.  When  planning  occurs  during 
development  of  the  first  variant,  commonality  projects  can  share  more 
effectively  than  one-way  reuse. 
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Such  thinking  leads  to  Building  Block  commonality,  which  occurs 
when  commonality  of  selected  high-value  systems  is  planned.  The  first 
variant  in  time  develops  a  common  building  block  that  takes  into  account 
the  needs  of  future  variants  and  will  be  used  in  those  future  variants. 
Building  Block  commonality  is  a  more  sophisticated  strategy  than  Reac¬ 
tive  Reuse  because  it  requires  a  trade  between  the  cost  of  developing  the 
building  block  and  the  life-cycle  savings  from  commonality. 

Widespread  Forward  commonality  occurs  when  commonality 
becomes  embedded  into  an  organization's  engineering  culture,  and 
each  design  decision  is  examined  for  its  commonality  implications.  In 
the  19  case  studies  examined,  Widespread  Forward  commonality  only 
occurred  when  one  corporation  tightly  controlled  the  development 
process.  Widespread  Forward  commonality  is  unlikely  to  be  the  right 
strategy  for  multicontractor  government  acquisition. 

The  implication  of  these  three  strategies  for  acquisition  is  that 
commonality  can  operate  in  three  significantly  different  modes,  and 
an  acquisition  strategy  appropriate  for  one  may  not  be  appropriate  for 
the  others. 

Synthesizing  a  Commonality  Process  Map 

After  reviewing  these  principles,  commonality  is  clearly  problem¬ 
atic  for  existing  acquisition  approaches.  For  example,  how  should  a 
contractor  be  incentivized  to  develop  a  common  system  where  directly 
measuring  commonality  is  not  a  good  reflection  of  the  needs  of  the 
customer?  How  can  programs  funded  year-to-year  “invest”  in  com¬ 
monality  for  future  cost  savings?  Does  the  emphasis  on  competition  in 
acquisition  allow  the  sort  of  cooperation  between  contractors  needed 
to  develop  common  systems? 

The  first  step  in  designing  an  effective  acquisition  strategy  is  to  be 
clear  about  the  steps  required  for  a  commonality  acquisition.  Figure  1 
lists  steps  that  represent  a  process  map  for  Building  Block  commonal¬ 
ity.  The  process  map  was  developed  by  examining  the  19  case  studies 
and  synthesizing  lessons  learned  in  the  commonality  projects  studied 
into  a  set  of  best-practice  steps.  The  process  maps  are  consistent  with 
the  analysis  of  the  case  studies  undertaken  by  Boas,  Hofstetter,  Rhodes, 
and  Cameron. 
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FIGURE  1.  PROCESS  MAP  FOR  BUILDING  BLOCK  COMMONALITY 
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Additional  explanation  of  how  the  process  maps  fit  the  observed 
performance  of  commonality  projects  is  presented  in  Wicht  (2011). 
Slightly  different  process  maps  for  the  Reactive  Reuse  and  Widespread 
Forward  strategies  were  also  developed.  Space  precludes  their  inclusion 
here;  however,  full  details  are  presented  in  Wicht  (2011),  along  with  a  full 
explanation  of  the  elements  of  the  process  map  and  the  tools  that  can  be 
used  to  undertake  the  processes. 

Figure  1  shows  a  process  that  consists  of  an  entry  gateway  followed 
by  three  interactive  processes:  Identify,  Evaluate,  and  Implement.  The 
entry  gateway  screens  commonality  opportunities  to  allow  only  those 
opportunities  suitable  for  Building  Block  commonality  into  the  process 
steps.  The  Identify  process  takes  the  engineering  environment  of  the 
potentially  common  systems  and  evaluates  the  technical  feasibility  and 
performance  penalty  of  using  common  systems  in  place  of  unique.  The 
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second  process,  Evaluate,  measures  the  benefits  and  drawbacks  to  the 
common  solution  under  the  assumed  use  case,  compares  those  with  the 
unique  solution,  and  results  in  a  decision  to  invest  in  the  common  build¬ 
ing  block  or  to  pursue  the  unique  solution  instead.  The  third  process 
is  Implement,  which  manages  divergence  and  requires  reexamination 
of  the  cost-benefit  analysis  undertaken  in  the  Evaluate  process  as  the 
estimated  costs  and  benefits  become  better  known.  The  management 
of  divergence  through  the  Implement  process  is  ongoing  throughout  the 
product  life  cycle. 

Designing  Acquisition  Strategies  for  Commonality 

Two  levels  of  acquisition  strategy  are  important  in  designing  most 
commonality  acquisitions.  The  family-level  acquisition  strategy  consid¬ 
ers  the  acquisition  strategy  applied  to  the  family  of  products  as  a  whole. 
It  examines  which  organizations  (if  any)  have  responsibility  for  man¬ 
agement,  systems  engineering,  and  design  trade-offs  across  the  whole 
family.  The  variant-level  acquisition  strategy  considers  the  acquisition 
strategy  at  the  level  of  the  systems  that  integrate  to  produce  a  variant. 
It  asks,  for  example,  whether  a  single  contractor  is  responsible  for  a 
single  system  across  all  variants,  or  whether  each  system  is  separately 
competed  on  each  variant. 

To  illustrate  these  differences,  a  simplified  example  from  the  JTRS 
case  study  is  presented  in  Figure  2.  The  top  level  (the  family-level  acqui¬ 
sition  strategy)  concerns  the  relationship  between  the  organization  or 
organizations  responsible  for  producing  Radio  Type  1  and  Radio  Type  2. 
Different  commonality  outcomes  could  be  expected  if  the  same  com¬ 
pany  was  responsible  for  both  radios  compared  to  multiple  competitors 
responsible  for  one  type  of  radio  each.  The  lower  level  (called  variant 
level)  in  this  example  asks  questions  such  as:  Should  the  transmitter  be 
separately  competed  for  each  radio?  Should  it  be  separately  competed 
by  the  government  and  supplied  as  Government  Furnished  Equipment 
(GFE)?  Or  should  both  transmitters  be  awarded  to  a  single  company? 
Again,  different  commonality  outcomes  could  be  expected  under  these 
different  acquisition  structures. 
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FIGURE  2.  DEFENSE  RADIO  PROGRAM 


Family  level:  how  should  these 
two  potentially  similar  radios  be  acquired? 
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The  variants  are  decomposed  into  their  function  because  Hofstetter 
(2009)  showed  that  the  first  gateway  for  technically  feasible  common¬ 
ality  is  delivery  of  a  common  function.  This  is  common  sense:  A  single 
company  developing  the  transmitters  for  two  radios  is  more  likely  to 
deliver  commonality  than  a  single  company  developing  the  transmitter 
for  one  and  the  user  interface  for  another. 

The  following  questions  will  introduce  and  explore  the  best 
approaches  for  structuring  the  acquisition,  first  at  the  family  level  and 
then  at  the  variant  level. 
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Which  Family-level  Acquisition  Structure  Should  Be  Used? 

Figure  3  displays  three  options  for  family-level  acquisition  struc¬ 
tures,  which  were  identified  through  the  research,  case  studies,  and 
interviews  described  in  this  section. 

•  Single  Total  System  Performance  Responsibility 
(TSPR)  Contractor.  This  approach  awards  a  single  con¬ 
tractor  the  responsibility  for  development  of  the  whole 
family,  often  referred  to  as  TSPR,  or  a  Lead  System  Integra¬ 
tor  (see  Flood  &  Richard  [2005]  or  Loudin  [2010]). 

•  Multiple  Contractors  plus  Systems  Engineering  and 
Technical  Assistance  (SETA).  The  government's  sys¬ 
tems  engineering  and  integration  capabilities  are  enhanced 
by  awarding  a  contract  for  SETA  to  a  separate  contractor. 

•  Multiple  Prime  Contractors.  This  approach  is  the  tradi¬ 
tional  avenue  of  acquisition  through  competition.  A  contract 
is  separately  competed  and  awarded  for  each  variant. 


FIGURE  3.  GOVERNMENT  INVOLVEMENT  IN  SYSTEMS 
ENGINEERING  AND  INTEGRATION 


j  TSPR 


LOW 


HIGH 


Each  strategy  was  analyzed  for  its  effect  on  commonality  by  sepa¬ 
rately  considering  how  well  each  process  step  shown  in  Figure  1  could 
be  undertaken  under  the  particular  acquisition  strategy.  The  three  case 
studies  and  17  additional  interviews  yielded  extensive  information  about 
how  each  of  the  acquisition  strategies  performed  in  helping  to  achieve  the 
key  commonality  processes  identified  in  the  process  maps.  A  table  was 
used  to  score  each  acquisition  strategy  at  the  family  and  variant  levels 
against  the  processes  required  by  best-practice  commonality  for  each 
strategy,  as  shown  in  the  process  maps  discussed  earlier.  Four  scores 
were  possible:  (a)  under  this  acquisition  strategy,  the  process  step  was 
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more  likely  to  be  achieved;  (b)  under  this  acquisition  strategy,  the  process 
step  was  less  likely  to  be  achieved;  (c)  under  this  acquisition  strategy, 
there  would  be  no  effect  on  achieving  the  process  step;  or  (d)  under  this 
acquisition  strategy,  the  likelihood  of  achieving  the  process  step  could 
increase  or  decrease,  depending  on  other  factors. 

Figure  4  presents  an  example  of  an  analysis  for  the  specific  case 
of  an  acquisition  strategy  using  a  directed  subcontractor  (basically, 
selecting  a  contractor  that  has  built  the  system  in  a  previous  variant 
without  a  competitive  process).  Each  of  the  three  commonality  strate¬ 
gies  (Reactive  Reuse,  Building  Block,  and  Widespread  Forward)  head 
a  pair  of  columns.  The  left-hand  column  describes  the  commonality 
process  steps  necessary  for  best-practice  commonality,  and  the  right- 
hand  column  contains  an  assessment  of  how  well  that  process  would 
be  performed  with  a  directed  subcontractor  acquisition  strategy,  color 
coded  by  the  four  possible  scores.  The  complete  set  of  tables  covering 
every  acquisition  strategy  is  detailed  in  Wicht  (2011).  The  analysis  is 
coarse,  but  this  level  of  detail  was  justified  because  it  revealed  enough  to 
draw  new  conclusions  about  how  acquisition  structures  for  commonality 
should  be  conducted. 

The  analysis  concluded  that  effective  family-level  acquisition  struc¬ 
tures  have  three  roles  in  commonality: 

•  They  provide  strong  systems  engineering  to  arbitrate  per¬ 
formance-affordability  trades  made  by  the  variants. 

•  They  provide  strong  management  to  resist  variant-level 
improvements  in  cost  or  performance  if  they  adversely 
affects  the  family. 

•  They  share  information  and  intellectual  property  between 
the  variants. 

However,  the  extent  to  which  the  acquisition  structures  achieve  this 
depends  on  the  strength  of  systems  engineering  within  the  govern¬ 
ment  program  office  and  the  force  of  intellectual  property  provisions 
within  the  contract. 


Defense ARJ,  July  2012,  Vol.  19  No.  3: 221-248 


234 


A  Publication  of  the  Defense  Acquisition  University 


http :  //w  w  w.  dau.  mil 


FIGURE  4.  EVALUATION  OF  DIRECTED  CONTRACTOR 
(VARIANT-LEVEL  STRATEGY) 
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Figure  5  shows  the  results  of  the  family-level  analysis  in  more  detail. 
The  first  conclusion  is  that  if  the  government  systems  engineering  capa¬ 
bilities  are  strong,  then  any  of  the  strategies  are  likely  to  be  successful. 
Factors  other  than  commonality  can  be  allowed  to  govern  the  choice  of 
family-level  acquisition  strategy,  and  attention  should  be  focused  instead 
on  the  variant-level  acquisition  strategy.  In  this  context,  “strong”  govern¬ 
ment  systems  engineering  includes  the  ability  to  assess  commonality 
benefits  and  drawbacks  across  the  whole  family  life  cycle;  the  ability  to 
communicate  requirements  from  interfacing  systems  across  the  whole 
family  to  the  team  developing  the  common  system;  and  the  capability  to 
resist  unjustified  variant-level  divergence  that  is  detrimental  to  family 
life-cycle  cost  and  performance. 


FIGURE  5.  THREE  ROLES  OF  COMMONALITY  IN  EFFECTIVE 
FAMILY-LEVEL  ACQUISITION  STRUCTURES 
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subcontractors. 

The  conclusions  to  be  drawn  from  the  family-level  analysis  shown  in 
Figure  5  are  fourfold.  First,  if  government  systems  engineering  is  weak, 
then  independent  systems  engineering  from  a  SETA  organization  is 
probably  preferable  to  adopting  a  TSPR  approach.  Second,  if  existing  (or 
generated)  intellectual  property  is  likely  to  be  involved  in  the  common 
elements,  then  the  rights  to  use  that  intellectual  property  throughout 
the  family  should  be  included.  Third,  the  applicability  of  a  family-level 
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structure  is  not  affected  by  the  type  of  commonality  that  will  be  imple¬ 
mented.  Reactive  Reuse,  Building  Block,  and  Widespread  Forward 
commonality  all  require  good  systems  engineering,  strong  management, 
and  effective  information  sharing.  However,  if  any  are  weak,  then  the 
more  sophisticated  commonality  approaches  should  be  ruled  out.  The 
program  should  implement  Reactive  Reuse  or  discard  commonality  as 
the  architecting  strategy. 

The  fourth  and  final  conclusion  revealed  by  the  analysis  was  that 
the  family-level  acquisition  strategies  are  not  strongly  coupled  with  the 
performance  of  the  variant-level  acquisition  strategies,  which  allows  the 
family-level  structures  and  the  variant-level  structures  to  be  evaluated 
separately.  The  variant-level  acquisition  strategies  are  examined  in  the 
following  section. 

Which  Variant-level  Structure  Should  be  Used? 

For  the  variant-level  structures,  six  possibilities  were  considered 
(Figure  6). 

•  Fully  competitive.  The  system  is  acquired  by  allowing  all 
qualified  bidders  to  submit  proposals  for  each  system  and 
choosing  the  best  system  independently  for  each  variant. 

•  Joint  venture.  A  joint  venture  between  two  organizations 
is  formed  to  build  two  systems,  when,  in  the  absence  of  the 
joint  venture,  the  organizations  would  have  built  one  each. 

•  Directed  contractor.  A  contractor  that  has  built  the 
system  in  a  previous  variant  is  selected  without  a  com¬ 
petitive  process. 

•  Long-term  supplier.  A  contractor  is  chosen  competitively 
as  the  sole  supplier  of  a  particular  system  across  all  variants. 

•  Build-to-print.  Detailed  system  specifications  are  provided 
by  the  government  for  contractors  to  build  to  on  each  variant. 

•  GFE.  A  completed  system  is  supplied  directly  to  a  contrac¬ 
tor  by  the  government. 
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FIGURE  6.  CHOOSING  A  VARIANT-LEVEL  POSSIBILITY  (SIX 
POSSIBLE  SCENARIOS) 


FULLY  COMPETITIVE 

System  on  System  on 

Variant  1  Variant  2 


Design 

Design 

Production 

Production 

Sequence: 

System  2  awarded  after  System  1 


JOINT  VENTURE  (JV) 

System  on  System  on 

Variant  1  Variant  2 


Design 

Design 

Production 

Production 

Sequence: 

JV  awarded  System  1  and 
System  2  simultaneously 


DIRECTED  CONTRACTOR 

System  on  System  on 

Variant  1  Variant  2 


Design 

Design 

Production 

Production 

Sequence: 

System  2  awarded  after  System  1 


LONG-TERM  SUPPLIER 

System  on  System  on 

Variant  1  Variant  2 


Design 

Design 

Production 

Production 

Sequence: 

Supplier  awarded  System  1  and 
System  2  simultaneously 


BUILD-TO-PRINT 

System  on 

System  on 

Variant  1 

Variant  2 

Design 

Design 

Production 

Production 

Sequence: 

Immaterial  as  design  is  fixed 


GFE 

System  on  System  on 

Variant  1  Variant  2 


Design 

Design 

Production 

Production 

Sequence: 

Government  plans  to  develop  both 
systems  from  the  outset 


KEY  TO  CONTRACT  RESPONSIBILITY 


Contractor  1 

Contractor  2 

Joint 

Venture  1  +  2 

Government 

GFE 

Contractor 

Not  all  of  the  system  acquisition  strategy  variant-level  structures  are 
equally  favored  within  the  acquisition  community.  For  example,  directed 
contractors  are  not  preferred  when  full  and  open  competition  is  avail¬ 
able,  but  if  sufficient  justification  exists,  then  a  sole-source  acquisition 
could  be  used: 

Agencies  acquiring  major  systems  shall...  (b)  sustain  effective 
competition  between  alternate  systems  and  sources  for  as 
long  as  is  beneficial.  (Federal  Acquisition  Regulation  [FAR] 
Subpart  34.002) 

Figure  7  shows  the  relative  support  for  each  system  acquisition 
strategy  variant-level  structure  within  the  acquisition  community, 
divided  into  Good,  Moderate,  and  Poor  support.  The  six  variant-level 
structures  are  then  examined  for  effect  on  the  commonality  processes. 
At  this  stage,  the  contract  is  assumed  to  simply  reflect  the  natural  incen¬ 
tives  of  its  structure,  without  specific  contract  terms,  which  will  be 
investigated  in  the  next  section.  Figure  8  summarizes  the  analysis  of 
the  variant-level  strategies. 
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FIGURE  7.  SUPPORT  WITHIN  ACQUISITION  COMMUNITY  FOR  EACH 
SYSTEM  ACQUISITION  STRATEGY  VARIANT-LEVEL  STRUCTURE 
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forced  on  the  market 
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Medium:  A  sole  source  justification  may  be  used  if  there  are 
good  reasons  to  do  so 
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Poor:  Long-term  supply  arrangements  are  difficult  to  justify 
under  the  FAR  on  decade-long  time  scales 

Build-to-Print 

Good:  A  common  strategy;  however,  the  downside  is  that  the 
solution  may  be  prematurely  constrained 

GFE 

Medium:  Although  used,  scoping  interviews  suggested  GFE 
raised  implementation  difficulties 

The  first  point  evidenced  from  Figure  8  is  that  the  variant-level  strat¬ 
egy  needs  to  be  matched  with  the  commonality  strategy.  For  example, 
a  directed  contractor  works  well  for  Reactive  Reuse,  but  poorly  for 
Building  Block  and  Widespread  Forward  commonality.  This  in  part 
explains  why  defense  projects  struggle  to  achieve  effective  commonality: 
the  acquisition  strategy  most  often  used  for  commonality  acquisition 
projects  in  the  defense  industry  is  Fully  Competitive,  which  performs 
moderately  well  for  Reactive  Reuse,  and  poorly  for  Building  Block  and 
Widespread  Forward  commonality. 

The  strategies  that  work  well  for  Reactive  Reuse  are  the  strate¬ 
gies  that  place  the  reused  system  and  the  system  to  be  developed  under 
one  contractor.  In  an  approximate  order  of  preference,  and  taking  into 
account  Figures  7  and  8: 

•  A  directed  contractor  is  a  good  strategy  because  the  contrac¬ 
tor  has  the  intellectual  property  and  practical  know-how  to 
reuse  its  previous  system.  There  is  clear  justification  for 
sole-sourcing  in  this  instance,  so  acquisition  regulations 
are  unlikely  to  be  problematic. 

•  A  joint  venture  between  the  contractors  who  are  to  develop 
the  two  systems  works  well  for  reuse;  however,  this  will  only 
be  appropriate  in  circumstances  where  the  joint  venture 
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FIGURE  8.  EFFECT  OF  CONTRACT  STRUCTURE  ALONE  ON 
COMMONALITY 
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for  additional  up-front 
cost  and  is  well  placed  to 
trade  up-front  cost  against 
life-cycle  affordability. 

Poor:  Structure  is  not 
responsive  to  divergence 
because  the  design 
and  manufacturing 
organizations  are 
separate.  Also 
difficult  to  set  up  and 
manage  each  time 
a  new  commonality 
opportunity  appears. 

GFE 

Medium 

Poor:  Places  onus 
of  investigating  and 
evaluating  commonality 
on  GFE  contractor 
that  does  not  have 
previous  expertise  (if  it 
does,  effectively  it  is  a 
directed  contractor). 

Good:  GFE  supplier  could 
develop  good  building 
block  so  long  as  design  is 
well  known  at  time  GFE 
contract  is  let.  Can  tolerate 
more  divergence  than 
Build-to-Print  because 

GFE  contractor  remains 
responsible  for  design  and 
can  evaluate  economic 
case  for  divergence. 

Poor:  Structure  is  not 
adaptable  because 
there  is  a  firm  boundary 
between  GFE  and 
non-GFE.  Commonality 
opportunities  across  this 
boundary  will  not  be 
implemented. 
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would  be  natural  in  the  market.  Incentivizing  the  joint 
venture  may  create  economic  responsibilities  for  the  gov¬ 
ernment  that  outweigh  commonality  savings. 

•  Creating  a  long-term  supplier  for  a  particular  system  works 
well  to  encourage  reuse.  The  disadvantage  is  that  it  is  dif¬ 
ficult  to  justify  under  acquisition  regulations  because  the 
long-term  supplier  must  obtain  a  contract  for  the  duration 
of  the  family  development,  which  for  many  acquisitions  may 
be  a  decade  or  more. 

The  strategies  that  work  well  for  developing  Building  Block  com¬ 
monality  are: 

•  A  Build-to-Print  strategy,  where  the  government  cre¬ 
ates  the  design  for  the  common  building  block,  which  is 
then  competitively  manufactured  for  each  variant.  This 
works  well  when  the  design  is  well  known  at  the  outset, 
and  when  divergence  is  likely  to  be  low— for  example,  in 
low-clockspeed  industries. 

•  A  GFE  strategy,  where  the  building  block  is  developed  and 
manufactured  by  the  government  (or  a  separate  contrac¬ 
tor  to  the  government)  and  supplied  to  each  variant.  This 
approach  is  more  tolerant  of  divergence  than  Build-to-Print 
because  the  government  can  manage  divergence  that  occurs 
as  a  result  of  learning  during  manufacturing,  and  could  be 
used  on  higher  technology  projects.  However,  both  govern¬ 
ment  and  contractors  expressed  aversion  to  GFE  projects 
due  to  programmatic  and  liability  risks. 

•  Creating  a  long-term  supplier  responsible  for  the  building 
blocks  on  an  ongoing  basis.  This  relieves  the  government  of 
responsibility  for  developing  the  building  block,  but  raises  the 
same  sole-sourcing  concerns  mentioned  in  Reactive  Reuse. 

The  poor  performance  of  strategies  on  Widespread  Forward  com¬ 
monality  reinforces  the  observation  made  in  Concept  5  that  it  is  an 
inappropriate  strategy  for  multicontractor  acquisitions.  If  Widespread 
Forward  were  to  be  used,  establishing  a  long-term  contractor  for  the 
system  across  all  variants  is  likely  to  be  the  most  successful  strategy. 
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Of  course,  an  acquisition  strategy  is  about  more  than  contract  struc¬ 
ture.  Several  examples  were  found  in  the  acquisition  case  studies  of  the 
same  subcontractor  producing  unique  designs  for  different  customers 
with  similar  needs.  The  provisions  of  the  contract  dealing  with  issues 
such  as  payment  structure  and  intellectual  property  affect  the  acquisi¬ 
tion  result,  and  were  investigated  in  detail  in  the  case  studies  and  scoping 
interviews.  Figure  9  summarizes  the  recommended  contract  additions 
for  those  strategies  that  were  graded  Medium  or  Good  in  Figure  8. 

In  Reactive  Reuse,  contracts  were  improved  through  the  use  of 
fixed-price  contracts  for  development  and  manufacture.  The  fixed- 
price  contract  incentivizes  reductions  in  the  up-front  development  cost, 
thereby  encouraging  reuse.  An  award  fee  based  on  thorough  investigation 
and  evaluation  of  commonality  may  also  help. 

Incentive  fees  may  also  be  considered  in  Reactive  Reuse,  especially 
if  there  will  be  benefits  to  the  government  through  the  life  cycle  from 
commonality,  not  just  a  reduction  in  up-front  cost.  Incentive  fees  should 
not  be  tied  to  fixed  levels  of  commonality,  for  example,  paying  a  fee  based 
on  the  percentage  of  commonality  achieved  because  this  discourages  an 
analysis  of  whether  a  particular  reuse  opportunity  is  net-beneficial.  It 
also  raises  very  practical  difficulties  in  assessing  whether  any  incentive 
should  be  paid  for  two  similar  parts.  Instead,  base  incentive  fees  on  a 
transparent  life-cycle  cost  model  if  one  is  available.  Basing  incentive  fees 
on  a  life-cycle  cost  model  developed  and  maintained  by  the  contractor 
should  be  avoided.  The  case  study  that  did  this  had  difficulty  establishing 
wide  confidence  in  the  model. 

An  additional  consideration  for  reuse  is  that  the  requirements  of 
the  contract  should  be  expressed  only  in  terms  of  minimum  acceptable 
performance  (though  incentive  fees  could  be  offered  for  improvements) 
so  that  trades  can  be  made  between  performance  and  affordability. 
Every  instance  of  reuse  examined  in  our  case  studies  involved  this  trade. 
Overconstraining  the  performance  specification  will  hamper  efforts  to 
implement  commonality. 

Finally,  consideration  should  be  given  to  intellectual  property  provi- 
sions.  The  contractor  should  be  given  relevant  access  to 
government-owned  intellectual  property  to  increase  the  range  and  qual- 
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FIGURE  9.  CONTRACTING  ADDITIONS  THAT  IMPROVE  VIABLE 
STRUCTURES 


System 

Acquisition 

Strategy 

Effect  of  Acquisition  Structure  on  Commonality  Process 

Reactive  Reuse 

Building  Block 

Widespread  Forward 
Commonality 
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Competitive 

Fixed-price  contract  to 
encourage  reuse.  Add 
incentive  fees  if  life-cycle  cost 
savings  from  commonality 
are  expected.  Improve 
contractor  knowledge  of 
reuse  opportunities  through 
a  domain-wide  knowledge 
base  and  strong  government 
intellectual  property  on 
previous  projects. 

Joint 

Venture 

Fixed-price  contract  to 
encourage  reuse.  Add 
incentive  fees  if  life-cycle  cost 
savings  from  commonality  are 
expected. 

Directed 

Contractor 

Fixed-price  contract  to 
encourage  reuse.  Add 
incentive  fees  if  life-cycle  cost 
savings  from  commonality  are 
expected. 

Long-term 

Supplier 

Fixed-price  contract  to 
encourage  reuse.  Add 
incentive  fees  if  life-cycle  cost 
savings  from  commonality  are 
expected. 

Cost-plus  contracts  to 
encourage  identification  of 
commonality  opportunities. 

Firm  requirements  across 
existing  and  future  systems. 

IP  provisions  that  allow 
supplier  switch  if  necessary 
to  avoid  monopoly.  Good 
government  understanding 
and  encouragement  that  up¬ 
front  costs  will  be  higher.  Add 
incentive  fees  if  life-cycle  cost 
savings  from  commonality  are 
expected. 

Lead/Follower  contracts 
to  keep  costs  low.  Strong 
system  engineering  and  cost 
modeling  to  support  life- 
cycle-based  incentives. 

Build-to- 

Print 

Good  government  insight 
into  previous  designs.  Strong 
government  negotiation  of 

IP  on  previous  projects  so 
government  has  technology 
to  reuse.  Good  government 
core  engineering  skills. 

Very  firm  requirements  across 
existing  and  future  systems. 

Good  government  core 
engineering  skills  in  the  initial 
design  phase. 

GFE 

Cost-plus  contracts  to 
encourage  identification  of 
commonality  opportunities. 

Add  incentive  fees  if  life- 
cycle  cost  savings  from 
commonality  are  expected. 

Firm  requirements  across 
existing  and  future  systems. 

Need  to  deal  with  liability  and 
programmatic  responsibility 
for  GFE. 
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ity  of  elements  that  the  contractor  may  reuse.  Consider  also  obtaining 
rights  to  the  new  intellectual  property  developed  in  the  design  to  allow 
unplanned  reuse  by  subsequent  designs. 

Building  Block  commonality  benefits  from  many  of  the  same  contract 
provisions.  The  required  performance  should  not  be  overcon¬ 
strained,  and  the  contractor  should  be  encouraged  to  trade  affordability 
and  performance. 

However,  the  payment  basis  to  the  contractor  should  be  different  for 
Reactive  Reuse.  A  cost-plus  contract  is  preferable  because  it  enables  the 
contractor  to  investigate  a  wider  range  of  commonality  opportunities 
and  develop  the  best  building  block,  even  if  it  costs  more  initially.  Incen¬ 
tive  payments  based  on  the  estimated  life-cycle  cost  of  the  building  block 
can  be  used  to  ensure  the  building  block  does  not  become  overdesigned. 
An  award  fee  and  close  supervision  by  a  government  systems  engineer¬ 
ing  team  will  further  reduce  the  risk  of  abuse  of  the  cost-plus  structure. 

The  intellectual  property  in  the  building  block  must  be  obtained  by 
the  government,  with  the  right  to  license  it  to  other  parties;  otherwise, 
the  government  risks  price  increases  by  the  building  block  contractor 
because  of  the  government's  high  cost  to  switch  contractors. 

Analysis  of  Acquisition  Regulations 

During  the  course  of  this  analysis,  the  FAR  and  DoD  5000.2  were 
closely  examined.  No  major  changes  were  considered  necessary  to 
improve  commonality  projects.  The  sections  that  permit  sole-sourcing 
adequately  cover  the  rationale  for  commonality  sole-sourcing,  for  exam¬ 
ple  FAR  6.302(l)(a)(ii). 

However,  several  trends  in  defense  acquisition  impact  commonal¬ 
ity  acquisition.  The  emphasis  on  open  architectures  (Rendon  &  Snider, 
2008,  p.  59)  is  in  tension  with  commonality  because  it  encourages  a 
proliferation  of  innovative  designs  rather  than  the  consistent  use  of  a 
single  design.  For  many  programs,  open  architectures  may  be  the  pre¬ 
ferred  solution,  but  it  is  important  to  recognize  that  commonality  and 
openness  are  mutually  exclusive  strategies.  The  trend  toward  greater 
use  of  commercial  off-the-shelf  products  is  synergistic  with  common¬ 
ality  because  it  encourages  the  same  performance-affordability  trades 
(and  can  be  seen  as  a  particularly  widespread  form  of  Reactive  Reuse). 
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Finally,  the  trend  toward  fully  funding  acquisitions  and  allowing  the 
Services  to  retain  amounts  saved  on  acquisitions  (Carter,  2010,  p.  3)  is 
likely  to  improve  commonality  outcomes  because  it  encourages  projects 
to  “invest”  in  common  building  blocks  over  time. 

Recommendations 

Five  recommendations  are  set  forth  as  a  result  of  this  research: 

1.  Defense  acquisitions  that  seek  to  use  commonality  to 
improve  affordability  must  integrate  the  commonality 
strategy  and  the  acquisition  strategy. 

2.  The  family-level  contract  and  management  structure  must 
be  built  around  a  strong  systems  engineering  team,  which 
has  the  vision  and  authority  to  force  variants  into  perfor¬ 
mance-affordability  compromises  that  achieve  value  at  the 
family  level. 

3.  At  the  variant  level,  traditional  competitive  procurement 
approaches  do  not  work  well  for  commonality,  and  sole¬ 
sourcing,  GFE,  and  Build-to-Print  approaches  should  be 
considered  instead. 

4.  The  payment  structure,  incentive  and  award  fees,  perfor¬ 
mance  specifications,  and  intellectual  property  provisions 
of  the  contract  must  all  be  considered  from  a  commonality 
viewpoint  for  a  successful  project. 

5.  No  changes  to  the  FAR  are  required  to  implement  effective 
commonality. 
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Conclusions 

Simply  transplanting  the  principle  of  commonality  from  commer¬ 
cial  product  development,  without  regard  to  the  different  approaches 
used  in  government  acquisition,  invites  disaster.  In  particular,  the 
government  acquisitions  with  the  most  to  gain  from  commonality  are 
those  with  a  mix  of  contractors  are  working  independently  on  projects 
that  overlap  significantly.  Commonality  is  not  implemented  over  such 
distributed  development  frameworks  in  commercial  development, 
and  government  acquisition  must  break  new  ground.  Understanding 
how  to  use  an  acquisition  strategy  to  incentivize  sensible  commonality 
between  companies  is  a  critical  step  in  allowing  commonality  to  realize 
its  affordability  promise. 
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Inserting  Agility  in 
System  Development 


Matthew  R.  Kennedy  and  Lt  Co!  Dan  Ware 


With  the  fast-paced  nature  of  technology,  rapidly  fielding 
systems  has  never  been  more  important.  Success 
depends  on  well-defined  requirements  and  the  ability  to 
rapidly  respond  to  change  during  and  after  deployment. 
The  inability  to  rapidly  respond  may  cause  the  system 
to  become  obsolete  before  initial  fielding.  Creating  a 
structure  where  processes  allow  for  changes  during 
system  development  requires  restructuring  system 
development  values  and  principles  at  all  levels.  This 
article  addresses  progress  toward  agility  and  defines 
agile  values  and  principles  being  used  by  agile  organi¬ 
zations  in  the  Business,  System,  and  Software  Aspects. 
It  also  defines  operationally  effective  agile  practices 
being  utilized  to  implement  those  values  and  principles 
that  provide  a  starting  point  for  inserting  agility  into  the 
system  development  process. 
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With  the  fast-paced  nature  of  technology,  the  need  to  rapidly  field 
systems  has  never  been  more  important.  Success  does  not  just  depend  on 
well-defined  requirements,  but  also  on  one's  ability  to  respond  to  change 
during  development,  deployment,  and  post-deployment.  The  inability 
to  rapidly  respond  to  change  may  cause  the  system  to  become  obsolete 
before  initial  fielding.  Creating  a  structure  where  processes  allow  for 
changes  to  occur  during  system  development  requires  a  restructuring 
of  system  development  values  and  principles  at  all  levels. 

Three  Aspects  of  a  Software 
Intensive  System  Development 

Software  Intensive  System  (SIS)  development  can  be  understood 
as  having  three  aspects:  Business,  System,  and  Software.  Although 
the  three  aspects  sometimes  overlap  one  another,  general  responsi¬ 
bilities  can  be  attributed  to  each.  The  Business  Aspect  is  responsible 
for  the  overall  acquisition  of  the  system,  including  contracting,  fund¬ 
ing,  operational  requirements,  and  overall  system  delivery  structure. 
Next,  the  System  Aspect  is  responsible  for  the  technical  and  technical 
management  aspects  of  the  system,  and  serves  as  the  interface  between 
management  and  engineers.  The  Software  Aspect  is  responsible  for  the 
software  items  contained  in  the  SIS.  Viewing  SIS  development  through 
the  lens  of  these  aspects  helps  highlight  components  of  the  work  that 
are  often  neglected. 

Agility  is  “the  speed  of  operations  within  an  organization  and  speed 
in  responding  to  customers  (reduced  cycle  times)"  (Massachusetts 
Institute  of  Technology,  n.d.).  It  must  be  incorporated  into  each  aspect. 
The  degree  of  agility  when  developing  an  Information  Technology  (IT) 
system  determines  the  organization's  ability  to  respond  to  change. 

Currently,  each  aspect  is  at  a  different  maturity  in  terms  of  the  agile 
frameworks  and  methodologies  available.  However,  the  speed  at  which 
changes  can  be  made  during  development  is  held  captive  by  the  aspect 
that  is  most  resistant  to  change.  This  article  addresses  each  aspect  and 
its  progress  toward  agility,  and  defines  the  agile  values  and  principles 
being  used  by  agile  organizations  in  both  the  Business  and  Software 
Aspects.  It  defines  agile  practices  being  utilized  to  implement  these 
values  and  principles  to  provide  a  starting  point  for  inserting  agility  into 
the  system  development  process. 
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Business  Aspect 

The  Business  Aspect  is  where  operational  requirements  are  realized 
and  the  strategy  for  overall  system  development  is  identified.  Currently, 
the  Department  of  Defense  (DoD)  uses  DoD  Instruction  (DoDI)  5000.02 
to  manage  how  it  will  perform  the  acquisition  of  weapon  systems,  ser¬ 
vices,  and  Automated  Information  Systems  (AIS)  (DoD,  2008). 

Recognizing  that  the  current  DoDI  5000.02  was  not  responsive  to 
the  changing  needs  of  technology,  Congress  signed  the  Fiscal  Year  2010 
National  Defense  Authorization  Act  (NDAA),  which  directed  the  Sec¬ 
retary  of  Defense  to  “develop  and  implement  a  new  acquisition  process 
for  information  technology  systems”  (NDAA,  2009).  This  new  Defense 
Acquisition  System  process  must  include: 

•  early  and  continual  involvement  of  the  user; 

•  multiple,  rapidly  executed  increments  or  releases  of 
capability; 

•  early,  successive  prototyping  to  support  an  evolutionary 
approach;  and 

•  a  modular,  open-systems  approach  (NDAA,  2009). 

Moreover,  this  process  should  be  based  on  the  March  2009  Report 
of  the  Defense  Science  Board  (DSB)  Task  Force  on  Department  of  Defense 
Policies  and  Procedures  for  the  Acquisition  of  Information  Technology 
(NDAA,  2009).  The  DSB  report  concluded  that  “the  conventional  DoD 
acquisition  process  is  too  long  and  too  cumbersome  to  fit  the  needs  of 
the  many  IT  systems  that  require  continuous  changes  and  upgrades” 
(DSB,  2009).  The  report  also  noted  that  an  agile  acquisition  approach 
would  increase  IT  capability  and  program  predictability,  reduce  cost, 
and  decrease  cycle  time. 

The  DSB  has  developed  an  Agile  Business  Aspect  framework,  which 
is  divided  into  four  phases:  Business  Case  Analysis  and  Development, 
Architectural  Development  and  Risk  Reduction,  Development  and  Dem¬ 
onstration,  and  Operations  and  Support  (DSB,  2009). 
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Figure  1  depicts  the  four  phases  of  an  Agile  Business  Aspect  Frame¬ 
work  (DSB,  2009).  A  brief  description  of  each  phase  follows: 

•  Business  Case  Analysis  and  Development:  “Establish 
the  need  for  the  proposed  capability  and  develop  the  concept 
for  the  proposed  solution  and  perform  a  cost-benefit  analy¬ 
sis  to  quantify  the  benefits  of  the  solution  .” 

•  Architectural  Development  and  Risk  Reduction:  “The 
core  architecture  is  built  and  architecturally  significant 
features  demonstrated.  Prototyping  begins  during  this 
phase  and  continues  throughout  the  acquisition  life  cycle 
to  assess  the  viability  of  technologies  and  minimize  high- 
risk  features.” 

•  Development  and  Demonstration:  “The  period  when 
operational  capability  is  built  and  delivered  for  a  discrete 
number  of  releases.  Capabilities  are  prioritized  and  parsed 
into  groupings  to  establish  release  baselines  for  the  sub¬ 
programs.  Includes  development  of  training  programs  and 
testing  in  realistic  environments  to  ensure  successful  field¬ 
ing  of  new  capabilities.” 

•  Operations  and  Support:  “Provides  materiel  readiness, 
user  training,  and  operational  support  over  the  total  pro¬ 
gram  life  cycle.” 

In  addition  to  the  emerging  IT  Acquisition  framework,  the  DoD 
developed  an  agile  requirements  process  for  IT  systems  called  the  “IT 
Box”  (Wells,  2009).  The  Joint  Requirements  Oversight  Council  Memo¬ 
randum  008-08  stated,  “IT  programs  are  dynamic  in  nature  and  have,  on 
average,  produced  improvements  in  performance  every  12-18  months” 
(Joint  Requirements  Oversight  Council,  2009).  Recognizing  the  need 
for  performance  improvements,  the  “IT  Box”  allows  IT  programs  the 
flexibility  to  incorporate  evolving  technologies.  This  allows  for  greater 
agility  in  the  current  DoD  requirements  process. 
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To  be  used  in  conjunction  with  the  framework  is  a  guiding  value  set 
called  FIST  (Fast,  Inexpensive,  Simple,  Tiny),  which  may  be  utilized 
throughout  the  process  (Ward,  2010).  The  FIST  approach  identifies  a  set 
of  priorities  and  preferences  that  should  be  employed  by  project  leaders 
during  the  development  process  to  streamline,  accelerate,  and  simplify 
(Ward,  2010).  These  values  are  declared  in  the  FIST  manifesto  as: 

Talent  trumps  process. 

Teamwork  trumps  paperwork. 

Leadership  trumps  management. 

Trust  trumps  oversight.  (Ward,  2010) 

The  FIST  Manifesto  also  contains  a  series  of  principles  and  imple¬ 
mentation  guidelines,  which  can  be  applied  to  all  three  aspects  of 
development  (System,  Software,  and  Business).  These  principles  follow: 

•  Fixed  funding  and  floating  requirements  are  better  than 
fixed  requirements  and  floating  funding. 

•  Complexity  is  cost. 

•  Simplicity  scales.  Complexity  does  not. 

The  implementation  guidelines  include: 

•  Minimize  team  size  and  maximize  team  talent. 

•  Incentivize  and  reward  underruns. 

•  Requirements  must  be  achievable  within  short  time  hori¬ 
zons.  (Ward,  2010) 

The  FIST  approach  describes  a  particular  pattern  of  decision  mak¬ 
ing  that  has  been  successfully  used  on  various  DoD  programs.  Recent 
examples  include  the  Marine  Corps  “Harvest  Hawk,”  which  incorporated 
a  gunship  modification  onto  a  C-130  airframe.  This  modification  was 
fielded  just  18  months  after  the  program  was  announced  (Axe,  2010). 
Similarly,  the  U.S.  Air  Force's  new  intelligence,  surveillance,  and  recon- 
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naissance  aircraft— the  MC-12W— flew  its  first  combat  mission  just 
6  months  after  the  contract  was  signed.  This  is  a  divergence  from  the 
typical  decade-long  weapons  system  program  and  shows  the  DoD  can 
deliver  inexpensive  systems  on  short  timelines. 

In  addition  to  rapidly  delivering  inexpensive  systems,  capabilities 
produced  by  using  the  FIST  approach  tend  to  outperform  more  expen¬ 
sive,  complex  systems  when  actually  fielded.  Examples  include  the  Air 
Force's  Condor  Cluster  supercomputer,  which  was  developed  for  one- 
tenth  the  cost  of  a  traditional  supercomputer  and  uses  one-tenth  the 
electricity  of  comparable  systems.  It  operates  at  500  TFLOPS  (Tera 
FLoating  point  Operations  per  Second),  making  it  the  fastest  supercom¬ 
puter  in  the  entire  DoD. 

The  Agile  Business  Aspect  framework  and  the  FIST  approach  are 
examples  of  how  the  Business  Aspect  is  making  advancements  toward 
becoming  more  agile  and  adaptive  to  changing  requirements,  which  is 
required  to  keep  pace  with  today's  rapidly  changing  environment. 

System  Aspect 

The  System  Aspect  addresses  the  technical  and  technical  man¬ 
agement  pieces  of  the  system  and  serves  as  the  interface  between 
management  and  engineers.  Utilizing  various  systems  engineering 
standards  and  guides,  operational  requirements  are  decomposed  into 
technical  requirements.  The  System  Aspect  holds  the  overall  responsi¬ 
bility  for  the  development  of  the  system  given  the  contractual,  schedule, 
and  fiscal  constraints  of  the  Business  Aspect. 

Though  the  systems  engineering  process  is  generally  portrayed  in  a 
waterfall-like  fashion,  the  systems  engineering  community  has  moved 
toward  an  incremental  delivery  approach.  The  (DAG)  identifies  incre¬ 
mental  development  as  a  capability  that  Defense  Acquisition  Guidebook 
that  “is  developed  and  fielded  in  increments  with  each  successive  incre¬ 
ment  building  upon  earlier  increments  to  achieve  an  overall  capability''. 
This  incremental  approach  relies  heavily  on  prototyping  and  allows  for 
technology  maturation  in  subsequent  releases  (DAU,  2010).  The  move 
toward  an  incremental  delivery  allows  the  systems  engineering  process 
to  better  adapt  to  change  than  the  waterfall-like  implementation.  How¬ 
ever,  with  the  rapid  rate  of  change,  the  incorporation  of  an  incremental 
model  alone  may  not  be  enough.  Currently,  no  agile  systems  engineering 
frameworks,  principles,  or  values  are  in  place  to  guide  the  System  Aspect. 
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Software  Aspect 

The  Software  Aspect  addresses  the  software  items  contained  in  the 
SIS.  Provided  a  set  of  requirements  from  the  System  Aspect,  the  Soft¬ 
ware  Aspect  creates  the  software  items  required  for  the  system. 

Software  development  has  been  on  a  continuous  process  improve¬ 
ment  track  for  decades.  Initially,  the  waterfall  software  development 
methodology  was  used,  where  software  was  developed  in  one  long  release 
cycle  (Royce,  1970,  pp.  1-9),  although  this  approach  was  described  as 
“risky  and  invites  failure.”  The  waterfall  software  development  meth¬ 
odology  provides  the  fundamental  steps  required  to  develop  software. 
However,  it  has  one  major  flaw  in  that  it  assumes  that  once  the  require¬ 
ments  process  is  complete,  the  requirements  will  remain  unchanged 
throughout  the  development  life  cycle.  This  assumption  rarely  holds 
true  in  practice  as  change  is  inevitable  in  all  large  software  projects 
(Sommerville,  2004). 

Long  waterfall-like  development  cycles  do  not  allow  for  require¬ 
ments  changes,  a  flaw  identified  by  Royce  in  his  original  paper.  Breaking 
software  development  cycles  into  a  series  of  increments  allows  one  to 
better  adapt  to  changing  requirements.  In  the  incremental  model,  an 
increment  is  a  potentially  shippable  piece  of  functionality.  Incremental 
delivery  allows  the  user  to  gain  value  from  a  portion  of  the  system  prior 
to  the  entire  system  being  released. 

Agile  Software  Development 

Though  seen  as  an  improvement  over  the  waterfall  software  develop¬ 
ment  methodology,  the  incremental  approach  has  several  disadvantages; 
namely,  the  majority  of  requirements  must  still  be  known  up-front  (U.S. 
Air  Force,  2003).  Agile  processes  have  emerged  to  match  the  pace  in 
which  change  is  encountered  during  software  development. 

Agile  software  development  is  a  broad  term  used  to  describe  devel¬ 
opment  methodologies  that  adhere  to  a  set  of  values  and  principles 
defined  by  the  Agile  Manifesto  (Beedle  et  al.,  2001).  The  Agile  Mani¬ 
festo  was  formed  when  a  group  of  12  people  calling  themselves  the  Agile 
Alliance  gathered  to  find  an  alternative  to  the  current  documentation- 
driven,  heavyweight  software  development  process  (Beedle  et  al.,  2001). 
Through  this  effort,  they  framed  the  following  set  of  values  to  improve 
the  way  software  is  developed  (Beedle  et  al.,  2001): 
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•  Individuals  and  interactions  over  processes  and  tools; 

•  Working  software  over  comprehensive  documentation; 

•  Customer  collaboration  over  contract  negotiation;  and 

•  Responding  to  change  over  following  a  plan. 

The  Agile  Manifesto  also  defines  the  following  principles,  which  are 
used  to  separate  agile  practices  from  their  heavyweight  counterparts 
(Martin  &  Martin,  2006): 

•  Our  highest  priority  is  to  satisfy  the  customer  through  early 
and  continuous  delivery  of  valuable  software. 

•  Welcome  changing  requirements,  even  late  in  development. 
Agile  processes  harness  change  for  the  customer's  competi¬ 
tive  advantage. 

•  Deliver  working  software  frequently,  from  a  couple  of  weeks 
to  a  couple  of  months,  with  a  preference  to  the  shorter 
timescale. 

•  Working  software  is  the  primary  measure  of  progress. 

•  Agile  processes  promote  sustainable  development.  The 
sponsors,  developers,  and  users  should  be  able  to  maintain 
a  constant  pace  indefinitely. 

•  Simplicity— the  art  of  maximizing  the  amount  of  work  not 
done— is  essential. 

The  application  of  these  principles  varies  in  practice  as  no  pre¬ 
determined  number  of  principles  must  be  utilized  for  a  development 
methodology  to  be  deemed  “agile.”  Several  development  methodol¬ 
ogies  are  in  use  today;  however,  a  survey  conducted  by  VersionOne, 
which  included  almost  1,700  individuals  and  71  countries,  found  Scrum 
and  extreme  Programming  to  be  the  most  widely  followed  method¬ 
ologies  (VersionOne,  2007).  Other  common  methodologies  include 
Crystal,  Dynamic  Systems  Development  Methodology,  and  Lean  Soft¬ 
ware  Development. 
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Scrum 

Scrum  is  a  framework  used  for  project  management,  which  is 
designed  for  projects  where  it  is  difficult  to  look  ahead  (Brede  Moe,  Ding- 
soyr,  &  Dyba,  2008,  pp.  76-85).  It  provides  a  framework  with  which  these 
activities  will  be  executed  (Figure  2).  Scrum  comprises  self-organizing 
and  self-managing  teams  that  release  a  potentially  shippable  product  in 
sprints  (increments)  of  2-4  weeks. 


FIGURE  2.  SCRUM  FRAMEWORK 


Note.  Adapted  from  The  SCRUM  process/SCRUM  framework  [Web  page],  by  Expert 
Program  Management  (n.d.)  at  http://www.expertprogrammanagement.com/2010/08/ 
the-scrum-process/. 


The  process  starts  with  a  product  backlog  (requirements)  that  is 
prioritized  by  the  user  prior  to  the  start  of  each  sprint.  The  team  then 
selects  what  can  be  accomplished  within  the  designated  sprint  duration; 
however,  the  team  must  select  the  requirements  in  the  order  specified  by 
the  user.  These  selected  requirements  then  become  the  sprint  backlog. 
The  items  on  the  sprint  backlog  are  what  will  be  delivered  to  the  cus¬ 
tomer  at  the  end  of  the  sprint. 
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extreme  Programming 

Whereas  Scrum  is  a  process  to  manage  a  product,  extreme  Program¬ 
ming  (XP)  is  an  agile  development  methodology  focused  on  software 
development  as  a  whole.  XP  is  one  of  the  most  well-documented  agile 
methodologies,  and  it  consists  of  the  following  12  rules  (Cohen,  Lindvall, 
&  Costa,  2003): 


1.  The  Planning 

Game 

2.  Small  Releases 

3.  System 

Metaphor 

4.  Simple  Design 

5.  Continuous 

Testing 

6.  Refactoring 

7.  Pair 

Programming 

8.  Collective  Code 

Ownership 

9.  Continuous 

Integration 

10.  40-Hour  Work 

Week 

11.  On-site 

Customer 

12.  Coding 

Standards 

No  set  number  of  rules  need  be  practiced  by  a  team  to  claim  they 
are  doing  XP  (Wolak,  2001).  However,  the  strength  of  XP  is  in  the  com¬ 
bination  of  the  rules  and  not  implementing  a  single  rule  alone  (Cohen, 
Lindvall,  &  Costa,  2003). 

The  Software  Aspect  has  a  greater  selection  of  agile  methodologies 
to  utilize  during  development,  allowing  for  valuable  resources  when 
inserting  agility  within  the  Software  Aspect. 

Maintaining  Agility  Between  Aspects 

With  the  growing  complexity  of  today's  systems,  the  systems  engi¬ 
neering  effort  becomes  increasingly  important  to  success.  Currently, 
both  the  Business  and  Software  Aspects  have  an  agile  framework  and  a 
proven  set  of  agile  values  to  help  guide  development.  However,  travers¬ 
ing  from  the  Business  Aspect  to  the  Software  Aspect  requires  passing 
through  the  System  Aspect,  which  could  hinder  the  agile  advances  made 
in  the  other  aspects.  The  System  Aspect's  ability  to  respond  to  the  agile 
processes  developed  within  the  Business  Aspect,  as  well  as  fostering 
the  agile  processes  in  the  Software  Aspect,  could  play  a  pivotal  role  in 
overall  system  success. 

Agile  Principles 

When  combining  the  FIST  implementation  guidelines  and  prin¬ 
ciples  and  comparing  them  against  similar  principles,  much  constancy 
is  evident. 
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Though  no  one-to-one  relationship  exists  between  the  FIST  prin¬ 
ciples/guidelines  and  the  Agile  Manifesto  principles,  they  all  remain 
important  complementary  principles  while  developing  a  complete  agile 
organization. 

Agile  Practices 

Agile  projects  use  various  practices  to  implement  the  Agile  Values 
and  Principles  identified.  When  considering  both  the  Software  and  Busi¬ 
ness  Aspects,  a  common  set  of  practices  emerges.  These  practices  are: 


Incremental  Development 

Small  Teams 

Iterative  Development 

Time  Boxing 

Short  Time-lines 

Lean  Initiatives 

Retrospectives  (Lessons  Learned) 

Prototyping 

Empowered/Self-organizing/ 
Managing  Teams 

Continuous  User  Involvement 

Prioritized  Product  Backlog 
(Requirements) 

Co-located  Teams 

Implementation  of  these  practices  varies  greatly  from  project  to  proj¬ 
ect.  Using  co-located  teams  as  an  example,  a  large  program  retrofitting 
military  aircraft  may  be  structured  in  a  way  to  have  the  teams  located 
on  the  same  installation  so  that  the  contracting,  development,  and  test¬ 
ing  activities  are  located  on  the  same  installation.  This  contrasts  with 
software  development  teams,  which  implement  the  practice  of  co-located 
teams  by  having  the  development  team  work  in  the  same  room. 

These  practices  are  well-documented  and  demonstrated  and  offer 
great  promise  for  helping  deliver  affordable  systems  that  are  available 
when  needed  and  effective  when  used.  By  implementing  these  proven 
practices,  we  can  increase  agility  with  the  Systems  Aspect. 

What  to  Expect  from 
Implementing  Agile  Practices 

Studies  have  been  conducted  over  the  last  decade  documenting  the 
results  when  utilizing  agile  practices.  Rally  Software  Development  Cor¬ 
poration  found  an  average  37  percent  decrease  in  time-to-market  and 
a  16  percent  increase  in  productivity  (Software  Engineering  Institute, 


261 


Defense  ARJ,  July  2012,  Vol.  19  No.  3 : 249-264 


Inserting  Agility  in  System  Development 


n.d.).  Findings  from  seven  individual  studies  found  a  benefit-to-cost, 
productivity,  and  quality  ranging  from  14  percent  to  93  percent  (Rico, 
2008).  The  averages  from  the  study  can  be  found  below: 

67  percent,  average  increase  in  productivity, 

65  percent  average  increase  in  quality,  and 

49  percent  improvement  in  cost  (Rico,  2008). 

Conclusions 

More  than  ever,  military  technology  programs  need  to  rapidly  field 
systems  within  tight  budget  constraints  and  still  maintain  an  ability  to 
respond  to  change.  The  Agile  approach  provides  a  useful  starting  point 
to  achieve  these  objectives  of  speed,  thrift,  and  agility. 

Inserting  agility  within  an  organization  is  a  journey,  not  a  desti¬ 
nation.  Agile  practices  that  work  for  one  organization  may  not  be  as 
effective  when  implemented  at  another  organization.  Conversely,  agile 
practices  found  effective  within  an  organization  last  year  may  no  longer 
be  as  effective  as  their  initial  implementation  due  to  external,  internal, 
or  personnel  changes.  These  changes  may  require  periodic  modification 
or  even  removal  of  practices  to  remain  competitive  in  today's  fast-paced 
world  of  IT.  It  is  not  a  single  practice  that  makes  an  organization  agile, 
but  a  combination  of  practices. 
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Identifying  Organizational 

Conflict  of  Interest: 

The  Information  Gap 


M.  A.  Thomas 


As  the  volume  of  government  contracting  increases,  so 
does  the  importance  of  monitoring  government  contrac¬ 
tors  to  guard  against  Organizational  Conflict  of  Interest 
(OCI).  For  contracting  officers  to  identify  OCIs,  they  must 
be  able  to  identify  the  relevant  business  interests  of  a 
contractor’s  affiliates.  This  information  may  be  private 
or  not  easily  obtained.  Using  newly  released  data  to 
develop  preliminary  visualizations  of  contractor  orga¬ 
nizational  structures  shows  the  organizational  structure 
of  many  contractors  to  be  complex  and  multinational. 
The  complexity  and  the  lack  of  easily  available  public 
information  make  it  very  unlikely  that  contracting  officers 
could  identify  OCIs  without  substantial  improvements 
in  government  data  collection. 
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As  government  has  come  to  rely  more  heavily  on  contractors  for 
goods,  services,  and  advice,  it  needs  to  ensure  that  procurement  remains 
competitive  and  that  contractor  performance  is  not  compromised  by 
outside  interests.  Organizational  Conflict  of  Interest  (OCI)  refers  to 
conflicts  that  arise  because  of  conflicting  incentives  of  contractors  due 
to  their  own  activities  or  the  activities  of  related  entities.  Government 
contracting  officers  are  required  to  identify  and  respond  to  possible  OCIs 
during  the  contracting  process.  The  current  policy  debate  focuses  on 
issues  such  as  the  definition  of  an  OCI,  the  objectives  of  government  in 
avoiding  or  mitigating  OCIs,  the  relevant  contractor  relationships  and 
activities  that  should  be  considered  when  identifying  an  OCI,  and  the 
appropriate  responses  of  contracting  officers  to  OCIs  once  identified 
(Guttman,  1977;  Taylor,  1983;  Taylor  &  Dickson,  1984;  Gordon,  2005; 
Szeliga,  2005;  Yukins,  2011). 

Little  attention  has  been  paid  to  the  question  of  how  contracting 
officers  are  to  obtain  the  information  necessary  to  identify  an  OCI  in  the 
first  place.  Identification  of  an  OCI  would  require  identification  of  those 
entities  considered  to  be  sufficiently  closely  related  to  the  contractor 
to  be  important  as  well  as  knowledge  of  the  relevant  activities  of  these 
related  entities. 

An  important  current  debate  involves  the  extent  to  which  the 
business  interests  of  a  contractor's  affiliates  should  be  imputed  to  the 
contractor  so  as  to  give  rise  to  a  conflict.  The  Federal  Acquisition  Regu¬ 
lation  (FAR)  Section  2.101  (General  Services  Administration,  2005) 
defines  “affiliates”  as  “associated  business  concerns  or  individuals  if, 
directly  or  indirectly— (1)  Either  one  controls  or  can  control  the  other; 
or  (2)  A  third  party  controls  or  can  control  both.”  While  control  could 
be  contractual,  the  discussion  has  centered  primarily  on  ownership 
relationships  that  link  companies  in  a  single  organizational  structure. 

In  practice,  however,  contracting  officers  have  few  means  of  learning 
the  organizational  structure  of  contractors.1  Even  the  problem  of  provid¬ 
ing  a  definitive  identification  of  contractors  is  one  that  the  government 
has  not  yet  solved.  It  has  even  less  information  about  the  contractual 
relationships  of  the  contractor  and  its  affiliates,  such  as  teaming  arrange¬ 
ments  or  subcontracting  relationships,  which  can  have  multiple  tiers. 
Even  if  the  relevant  business  entities  were  identified,  the  government 
has  no  way  of  identifying  their  relevant  activities  or  financial  interests. 
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This  article  explains  the  information  gap  that  prevents  effec¬ 
tive  implementation  of  OCI  policy  and  focuses  in  particular  on  the 
opacity  of  contractor  organizational  structures.  An  analysis  of  newly 
available  data  from  Usaspending.gov  suggests  the  complexity  of  the 
organizational  structure  of  contractors  and  the  failure  of  government 
policymaking  to  adapt. 

Organizational  Conflict  of  Interest 

The  government  is  continuing  to  develop  and  articulate  its  policy 
on  OCIs,  including  the  definition  of  an  OCI,  the  government's  objectives 
in  identifying  and  responding  to  OCIs,  the  ways  in  which  contracting 
officers  should  identify  OCIs,  and  the  appropriate  response  for  contract¬ 
ing  officers  after  having  identified  an  OCI.  In  2010,  the  Defense  Council 
advanced  a  proposed  rule  to  update  the  Defense  Federal  Acquisition 
Regulation  Supplement  (proposed  DFARS  rule)  that  would  have  effec¬ 
tively  codified  existing  U.S.  Government  Accountability  Office  (GAO) 
case  law  on  OCIs  (Papson,  Doyle,  &  Ginsberg,  2011;  Defense  Federal 
Acquisition  Regulation  Supplement  [DFARS],  2010,  April).  Under  criti¬ 
cism,  it  retreated  from  certain  key  provisions  of  the  proposed  rule  in  its 
final  rulemaking,  awaiting  a  broader  revision  of  the  rules  on  OCI  by  the 
Federal  Acquisition  Regulation  Council  (DFARS,  2010,  December).  In 
2011,  the  FAR  Council  published  a  proposed  rule  (“proposed  FAR  rule”) 
that  offered  an  alternative  model  (FAR,  2011). 

A  key  policy  question  is  the  extent  to  which  the  business  interests  of 
other  related  entities,  and  in  particular  affiliates  related  by  ownership 
interests,  should  be  imputed  to  the  contractor.  A  series  of  prior  decisions 
by  the  GAO  had  affirmed  that  “all  business  interests  within  the  larger 
corporate  enterprise  are  imputed  to  every  entity  and  person  within 
the  enterprise”  (Papson  et  al.,  2011,  p.  2;  Comptroller  General,  1995). 
Accordingly,  in  2010  the  Defense  Acquisition  Regulations  Council  issued 
a  proposed  rule  that  defined  a  contractor  as  “a  party  to  a  government 
contract  other  than  the  government  and  includes  the  total  contractor 
organization,  including  not  only  the  business  unit  or  segment  that  signs 
the  contract.  It  also  includes  all  subsidiaries  and  affiliates”  (DFARS, 
2010,  April,  p.  20958).  The  proposed  FAR  rule  does  not  have  this  defini¬ 
tion  of  a  contractor,  but  defines  an  OCI  with  respect  to  the  relationship 
between  contractors  and  their  affiliates  (FAR,  2011,  p.  23242). 


Defense  ARJ,  July  2012,  Vol.  19  No.  3: 265-282 


268 


A  Publication  of  the  Defense  Acquisition  University 


http :  //w  w  w.  dau.  mil 


The  Role  of  Information 

To  identify  an  OCI,  contracting  officers  would  have  to  identify  the 
affiliates  of  a  contractor  and  their  relevant  financial  and  business  inter¬ 
ests.  At  present,  the  FAR  Section  9.506  enjoins  contracting  officers  to 
do  the  following: 

...  seek  the  information  from  within  the  Government  or  from 
other  readily  available  sources.  Government  sources  include 
the  files  and  the  knowledge  of  personnel  within  the  contracting 
office,  other  contracting  offices,  the  cognizant  contract  adminis¬ 
tration  and  audit  activities  and  offices  concerned  with  contract 
financing.  Non-Government  sources  include  publications  and 
commercial  services,  such  as  credit  rating  services,  trade  and 
financial  journals,  and  business  directories  and  registers.  (FAR, 
2005,  p.9.5-3) 

The  provision  in  the  proposed  FAR  rule  is  similar,  providing  that 
the  “contracting  officer  should  seek  readily  available  information  about 
the  financial  interests  of  the  offerors,  affiliates  of  the  offerors,  and  pro¬ 
spective  subcontractors  from  within  the  government  or  from  other 
sources  and  compare  this  information  against  information  provided  by 
the  offeror”  (FAR,  2011,  p.  23247).  The  proposed  FAR  rule  also  provides 
explicit  language  for  the  contracting  officer  to  include  in  the  solicitation 
to  require  contractors  to  disclose  information  regarding  potential  OCIs 
if  the  contracting  officer  has  determined  that  the  nature  of  the  contract 
is  such  that  an  OCI  might  arise  from  contract  performance  (FAR,  2011, 
p.  23239). 

Contracting  officers  are  not  likely  to  be  able  to  assess  the  financial 
interests  and  activities  of  affiliates  or  prospective  subcontractors  using 
readily  available  sources,  and  while  competitors  may  have  an  incentive 
to  bring  information  about  a  contractor's  OCI  to  the  attention  of  a  con¬ 
tracting  officer,  it  remains  questionable  whether  competitors  are  in  fact 
much  better  positioned  to  do  so.  The  offerors  themselves  may  lack  this 
information.  The  DFARS  proposed  rule  had  provided  that,  where  the 
contracting  officer  has  determined  that  the  nature  of  the  contract  is  such 
that  an  OCI  might  arise  from  contract  performance,  the  contractor  must 
describe  any  other  work  performed  by  itself  or  its  affiliates  within  the  past 
5  years  that  is  associated  with  the  offer  it  plans  to  submit  (DFARS,  2010, 
April,  p.  20957).  The  Coalition  for  Government  Procurement,  an  associa¬ 
tion  of  300  contractors,  argued  that  this  requirement  would  “have  the 
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unintended  consequence  of  driving  contractors  that  lack  sophisticated 
tracking  systems  [to  track  sales  of  commercial  items  and  services]  out  of 
the  marketplace”  (M.  Vakerics,  personal  communication,  July  21, 2010).2 

Indeed,  contracting  officers  may  be  hard  pressed  to  identify  affiliates 
in  the  first  place.  Information  on  the  organizational  structure  of  contrac¬ 
tors  is  not  always  in  the  public  domain. 


The  Available  Information  on  Contractors’ 
Identities  and  Complexities  of  Contractor 
Organizational  Structures 

Contractors’  Identities 

To  identify  OCIs,  contracting  officers  must  first  know  who  the  com¬ 
panies  are  that  contract  with  the  government.  Because  company  names 
may  not  be  unique,  because  a  single  business  can  operate  under  a  variety 
of  names,  and  because  locations  can  change,  this  requires  that  contrac¬ 
tors  be  given  unique  identifiers.  Since  1998,  the  government  has  used  a 
number  issued  by  the  private  firm  Dun  &  Bradstreet  (“D&B”)  to  identify 
government  contractors.3  A  business  that  wishes  to  contract  with  the 
government  gives  D&B  its  legal  business  name  and  physical  address, 
and  receives  a  nine-digit  Data  Universal  Numbering  System  number 
(“D-U-N-S”  or  “DUNS”  number).  The  DUNS  number  is  “a  unique  global 
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identifier  attached  to  operating  entities;  the  D-U-N-S  Number  is  never 
reassigned  to  another  company,  in  any  place,  at  any  time”  (Dun  and 
Bradstreet,  n.d.a).  A  different  DUNS  number  is  required  for  every  busi¬ 
ness  location  or  co-located  subdivision.  Under  FAR  section  4.11,  the 
contractor  then  uses  its  DUNS  number  to  register  in  the  Central  Con¬ 
tractor  Registration  (CCR)  database  maintained  by  the  Department  of 
Defense.  The  CCR  relies  upon  D&B  to  notify  the  CCR  of  any  changes  to 
the  contractor's  business  name  or  address. 

Millions  of  business  locations  have  DUNS  numbers  because  the  D&B 
identification  system  is  widely  used.  Dun  &  Bradstreet  has  assigned 
DUNS  numbers  to  more  than  100  million  companies  (Dun  and  Bradstreet, 
n.d.b).  However,  not  every  business  has  a  DUNS  number.  The  Excluded 
Parties  List  System,  the  government's  tool  for  identifying  debarred  com¬ 
panies,  warns  that  not  all  debarred  firms  have  DUNS  identifiers. 

In  addition  to  identifying  the  company,  contracting  officers  must 
identify  the  affiliates  of  the  contractor.  Company  organizational  struc¬ 
tures  can  be  opaque  even  for  companies  incorporated  in  the  United 
States.  While  the  Securities  and  Exchange  Commission  requires  pub¬ 
licly  traded  companies  to  disclose  some  types  of  information,  such  as 
ownership  and  purchase  and  sale  of  stocks  (Securities  and  Exchange 
Commission,  2009),  there  is  little  legal  requirement  for  disclosure  for 
businesses  that  are  not  publicly  traded.  In  2006,  the  GAO  testified  before 
the  Senate  that  in  the  process  of  incorporation,  minimal  ownership 
information  is  collected  (GAO,  2006).  The  GAO  reported  that  “[m]ost 
states  do  not  require  ownership  information  at  the  time  a  company  is 
formed  or  on  the  annual  and  biennial  reports  most  corporations  and  lim¬ 
ited  liability  companies  (LLC)  must  file''  (GAO,  2006).  Even  when  states 
do  collect  such  information,  they  do  not  verify  it.  As  a  consequence,  there 
maybe  no  publicly  available  information  on  the  organizational  structure 
of  a  private  business.  The  difficulty  of  identifying  organizational  struc¬ 
tures  is  such  that  when  a  company  is  debarred  from  federal  contracting 
because  of  misbehavior,  the  debarment  does  not  extend  to  wholly  owned 
subsidiaries,  in  large  part  because  the  government  has  no  way  to  identify 
them  (Governmentwide  Debarment  and  Suspension,  2003,  p.  66538). 

Nor  is  this  data  routinely  collected  when  a  contractor  registers 
prior  to  bidding  on  a  government  contract.  When  contractors  enter  the 
D&B  website  to  register  for  a  DUNS  number  as  required  under  the  FAR, 
they  may  optionally  enter  information  about  their  parent  company.  No 
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DUNS  number  is  assigned  to  the  parent  company  in  this  process,  and  so 
the  parent  company  may  not  have  a  unique  identifier.  Further,  there  is 
no  provision  for  entering  multiple  parents,  where  a  contractor  is  a  joint 
venture.  When  the  contractor  logs  into  the  CCR  with  its  DUNS  number, 
the  CCR  collects  substantial  information  that  includes  the  number  of  its 
employees  and  annual  receipts,  including  affiliates,  to  determine  if  the 
contractor  is  a  small  business.  It  does  not,  however,  collect  any  informa¬ 
tion  about  the  contractor's  organizational  structure  (Central  Contractor 
Registration,  2011). 

Both  the  data  identifying  the  contractor— the  DUNS  number— and 
whatever  data  links  the  contractor  to  its  parent  company  are  claimed  by 
D&B  as  private  property  even  though,  in  the  case  of  government  contrac¬ 
tors,  D&B  acquired  the  data  as  a  consequence  of  a  monopoly  established 
by  federal  regulation.  D&B  bundles  and  sells  corporate  information  and 
analysis  through  a  la  carte  reports  or  through  institutional  subscrip¬ 
tions— including  to  the  U.S.  government.  Among  Dun  &  Bradstreet's 
analytic  products  is  the  Corporate  Family  Tree  Plus,  which  allows  the 
user  to  get  information  about  the  affiliates  of  a  company  (D&B  Marketing 
Solutions,  n.d.).  Some  contracting  officers  may  have  access  to  the  Corpo¬ 
rate  Family  Tree  product  to  investigate  the  organizational  structure  of 
contractors,  but  it  is  not  universally  available  and  subscriptions  are  costly. 

Recently,  some  data  linking  contractors  to  parents  have  become 
publicly  available  through  a  government  transparency  initiative.  The 
government  has  been  engaged  in  a  decades-long  process  to  collect, 
centralize,  standardize,  improve  the  quality  of,  and  make  available 
procurement  data  (Acquisition  Advisory  Panel,  2007).  The  Federal 
Funding  Accountability  and  Transparency  Act  of  2006  (Transpar¬ 
ency  Act)  mandated  that  by  2008  the  Office  of  Management  and  Budget 
(OMB)  would  establish  a  single  searchable  and  freely  available  website 
that  included  basic  information  on  awards  of  federal  contracts.  This 
information  includes  the  name  and  location  of  the  entity  receiving  the 
award  and  “a  unique  identifier  of  the  entity  receiving  the  award  and  of 
the  parent  entity  of  the  recipient,  should  the  entity  be  owned  by  another 
entity"  (Federal  Funding  Accountability  and  Transparency  Act  of 2006, 
p.  120).  Shortly  thereafter,  the  government  established  the  website 
Usaspending.gov,  offering  a  user-friendly  interface  that  allows  the  public 
to  search  a  database  of  government  contracts,  to  view  summary  statis¬ 
tics,  or  to  download  raw  data  directly. 
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The  data  quality  problems  that  have  plagued  earlier  incarnations  of 
this  database  have  not  been  resolved.  Both  the  comprehensiveness  and 
the  quality  of  the  data  it  offers  have  been  criticized  (see,  e.g.,  Lee,  2011). 
Moreover,  the  OMB  has  not  yet  complied  with  the  legislative  requirement 
that  the  parents  of  contractors  be  listed  and  identified  by  unique  identi¬ 
fiers.  It  cannot,  given  that  the  government  neither  collects  information 
on  parents  nor  assigns  them  identifiers. 


In  2009,  Dun  &  Bradstreet  decided  to  allow  Usaspending.gov  to 
release  data  linking  contractors  to  their  parent  companies,  which  it 
had  “protected  in  [the  Dun  &  Bradstreet]  licensing  relationship  since 
its  inception”  (B.  William,  personal  communication,  October  20, 2009).4 
However,  there  are  still  consequences  for  the  government's  reliance 
on  third  party  data.  The  government  has  no  control  over  the  data  qual¬ 
ity.  D&B  does  not  have  the  data  that  the  government  must  supply  by 
law  because  it  does  not  require  contractors  to  supply  information  on 
their  parents  or  assign  parents  a  DUNS  number.  The  property  rights 
asserted  by  D&B  also  limit  the  use  of  the  data  that  it  does  have.  The 
Usaspending.gov  website  contains  a  disclaimer  titled  “Limited  Liability,” 
which  states  that  some  of  the  data  provided  “is  the  intellectual  property 
of  the  third  party  information  suppliers,”  is  supplied  without  any  kind 
of  warranty,  is  for  internal  use  only,  cannot  be  used  for  commercial  or 
marketing  purposes,  and  prohibits  “systematic  access”  or  extraction  of 
content  from  the  website. 
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Complexities  of  Contractor  Organizational  Structures 

While  the  quality  of  data  provided  on  Usaspending.gov  linking  con¬ 
tractors  to  parent  companies  is  poor,  analysis  of  that  data  suggests  that 
a  significant  number  of  contractors  may  have  complex  organizational 
structures.  Using  the  Contractor  Network  Extraction  Software  (ONES) 
written  by  the  author  to  analyze  the  Usaspending.gov  data,  it  was  pos¬ 
sible  to  reconstruct  part  of  the  organizational  structures  of  contractors 
by  matching  parents  and  subsidiaries  using  either  the  DUNS  number  or, 
where  this  is  lacking,  the  company  name.  This  in  turn  allows  analysis  at 
the  organizational  level  as  well  as  visualization  of  these  organizations 
using  freely  available  social  network  analysis  software. 

CNES  is  a  do  file  that  runs  under  STATA,  a  data  analysis  and  sta¬ 
tistical  software  package.  The  program  treats  each  Usaspending.gov 
record  containing  the  contractor  DUNS  number  (“dunsnumber”)  and 
parent  DUNS  number  (“parentdunsnumber”)  as  an  edge  in  a  directed 
graph  whose  nodes  are  DUNS  entities  (businesses  or  co-located  subdi¬ 
visions).  The  program  then  breaks  the  data  into  separate  components 
by  traversing  each  graph  and  assigning  a  common  identifier  (“com- 
ponentnum”)  to  each  edge  in  the  same  graph.  The  user  can  then  use 
STATA  or  CNES  utilities  to  select  components  of  interest  (for  example, 
components  containing  a  particular  business  name  or  components  of  a 
particular  size),  and  export  them  to  Netdraw  or  Pajek  for  visualization. 
Under  optional  name-based  matching,  the  program  will  match  edges 
based  on  the  contractor  name  (“recipientorcontractorname”)  and  parent 
name  (“parentrecipientorcontractorname”)  if  DUNS  numbers  are  not 
available.  Whether  an  edge  has  been  matched  based  on  DUNS  number 
or  name  is  preserved  in  the  variable  “pnamematch,”  and  exported  as  a 
tie  strength  variable  for  Netdraw,  which  allows  the  user  to  see  the  basis 
for  the  match  in  the  visualization  of  the  component.  This  is  important 
because  name-based  matching  is  more  error-prone  than  matching  based 
on  DUNS  numbers.  Readers  who  wish  more  information  are  invited  to 
consult  the  program  source  code  and  the  program  documentation,  which 
are  freely  available  under  a  GNU  General  Public  License  at  http://www. 
usgcontractors.info. 

The  Figure  shows  a  Netdraw  visualization  of  the  organizational 
structure  of  three  large  government  contractors  based  on  2010 
Usaspending.gov  data.  Each  node  represents  a  DUNS  entity  (a  location 
or  co-located  subdivision  of  a  business)  or  is  a  placeholder  for  a  parent 
that  is  named  in  the  dataset,  but  whose  DUNS  number  is  not  given. 
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FIGURE.  VISUALIZATION  OF  THREE  LARGE  GOVERNMENT 
CONTRACTORS  USING  CONTRACT  NETWORK  EXTRACTION 
SOFTWARE  (CNES) 


Note.  Visualization  of  three  large  government  contractors— SAIC,  Inc.,  Northrup 
Grumman  Corporation,  and  L-3  Communications  Holdings— joined  in  a  single  network 
perhaps  through  joint  ventures  or  transfer  of  business  units.  Adapted  from  2010 
Usaspending.gov  data  using  the  author’s  Contract  Network  Extraction  Software; 
visualized  using  Netdraw  (Borgatti,  2006).  Each  node  is  a  possible  location,  subdivision, 
or  subsidiary.  The  color  and  size  of  nodes  indicate  “degree,”  or  the  number  of  other 
nodes  to  which  it  is  connected. 


The  contractor  networks  produced  by  this  method  must  be  treated 
as  hypotheses  that  remain  to  be  confirmed  by  other  means  because  the 
data  quality  is  poor.  The  quality  and  timeliness  of  the  parent  linkage 
data  are  unknown— such  relationships  are  very  fluid,  and  it  is  not  clear 
if  there  is  any  auditing  to  ensure  the  correctness  of  data  entered  in  these 
fields.  Joint  ventures  are  reported  inconsistently,  and  all  parents  may 
not  be  listed. Some  entities  may  have  multiple  DUNS  numbers  and  use 
them  inconsistently.  Name-based  matching  risks  erroneous  matches  if 
companies  have  the  same  name,  as  well  as  the  risk  of  mistakenly  treating 
the  same  company  as  two  different  companies  because  of  variations  in 
the  entry  of  the  company  name  (although  the  program  does  control  for 
the  most  common  variations).  Finally,  because  Usaspending.gov  only 
contains  data  on  contractors  and  their  parents,  the  data  do  not  include 
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parts  of  the  organizational  structure  that  are  neither  contractors  nor  the 
parents  of  contractors.  Accordingly,  the  networks  produced  are  neces¬ 
sarily  fragmented  and  partial. 

Notwithstanding,  the  analysis  suggests  the  complex  organizational 
structure  of  an  important  percentage  of  government  contractors.  Ana¬ 
lyzing  the  2010  data  from  Usaspending.gov,  roughly  10  percent  of  the 
166,000  contractor  organizational  structures,  or  about  17,000  organiza¬ 
tions,  have  seven  or  more  related  locations  or  subsidiaries,  while  about 
6  percent  have  20  or  more.  Locations  and  subsidiaries  can  be  nested 
several  levels  deep,  and  some  organizations  are  multinational.  Because 
these  structures  are  partial,  more  complete  data  would  likely  show  a 
greater  level  of  complexity. 

Many  individual  companies  contract  across  a  range  of  government 
agencies,  which  suggests  that  any  process  for  gathering  information  on 
contractor  organizational  structures  must  be  located  at  a  governmental, 
rather  than  an  agency  level.  For  example,  in  2010  Oshkosh  Corpora¬ 
tion  contracted  with  agencies  including  the  Department  of  Defense, 
the  Department  of  Interior,  the  Department  of  Justice,  the  Department 
of  Homeland  Security,  the  General  Services  Administration,  and  the 
National  Aeronautics  and  Space  Administration.  But  even  companies 
that  contract  with  only  a  single  agency  may  belong  to  organizations  that 
contract  more  widely.  For  example,  as  foreign  aid  has  been  militarized 
over  the  last  decade,  a  number  of  aid  contractors  have  been  bought  by 
defense  contractors.  The  aid  contractors  continue  to  contract  only  or 
principally  with  the  United  States  Agency  for  International  Develop¬ 
ment,  but  their  organizations  contract  with  other  government  agencies. 

Given  this  level  of  complexity,  even  if  the  OMB  complied  with  the 
Transparency  Act  obligation  to  identify  the  parent  of  the  contractor, 
the  objective  of  allowing  the  public  to  understand  who  is  ultimately 
benefiting  from  a  government  contract  would  not  be  met.  Similarly,  this 
complexity  and  the  lack  of  easily  available  public  information  make 
it  very  unlikely  that  contracting  officers,  competitors,  the  public,  or 
even  contractors  themselves  could  identify  OCIs  without  substantial 
improvements  in  government  data  collection. 
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Conclusions 

Policy  debates  continue  about  how  government  contracting  officers 
should  best  handle  OCIs  when  they  encounter  them,  but  the  government 
does  not  have  and  is  not  collecting  the  information  necessary  to  detect 
OCIs  in  the  first  place.  While  the  FAR  lists  a  number  of  ways  for  con¬ 
tracting  officers  to  detect  OCIs,  including  asking  other  people  in  their 
offices,  these  methods  are  very  unlikely  to  result  in  detection  given  the 
complexity,  opacity,  and  international  character  of  many  organizational 
relationships. 

Ownership  relationships  are  not  the  only  type  of  relationships  that 
could  generate  an  organizational  conflict  of  interest,  but  detection  of 
OCIs  based  on  other  types  of  relationships  is  even  more  difficult.  Orga¬ 
nizations  may  have  contractual  relationships  that  could  give  rise  to 
conflicts,  such  as  teaming  and  subcontracting  relationships,  and  sub¬ 
contracting  relationships  can  be  tiered  several  layers  deep.  Neither  the 
government  nor  the  public  has  good  access  to  information  about  these 
relationships.  Usaspending.gov  has  started  making  available  informa¬ 
tion  on  first-tier  subcontracts,  but  without  the  information  needed  to 
link  them  to  their  primes.  Organizations  can  also  be  characterized  by 
interlocking  ownerships  where,  although  companies  are  legally  sepa¬ 
rate,  they  are  owned  or  managed  by  the  same  individuals.  The  GAO  has 
pointed  to  several  cases  in  which  owners  of  debarred  firms  continued  to 
receive  government  awards  by  spinning  off  new  companies  or  disguising 
the  true  owner  of  the  company  (GAO,  2009).  Identifying  interlocking 
ownership  could  not  be  accomplished  without  a  unique  identifier  for 
people— and  the  United  States  has  firmly  rejected  the  idea  of  creating 
such  an  identifier  out  of  privacy  concerns  and  fear  of  giving  too  much 
power  to  the  government  (see  Electronic  Privacy  Center,  2008).  Finally, 
the  question  of  how  to  identify  the  relevant  activities  of  related  entities 
remains  unanswered.  If  the  contractors  themselves  cannot  do  it,  it  seems 
very  unlikely  that  anyone  else  can. 

When  it  comes  to  OCI,  policymaking  is  outstripping  the  realities  of 
available  information.  In  the  absence  of  adequate  information,  the  policy 
debates  on  OCI  avoidance  and  mitigation  risk  being  largely  theoretical. 
The  questions  of  what  kind  of  information  is  needed  to  identify  OCIs, 
who  should  collect  this  information,  and  who  should  have  access  to  it 
must  be  addressed  in  the  elaboration  of  OCI  policy.  At  the  same  time, 
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the  need  for  information  about  the  identity,  relationship,  and  activities 
of  government  contractors  is  part  of  a  much  larger  discussion  regarding 
the  balance  between  security,  liberty,  privacy,  protection  from  misuse 
of  government  power,  and  the  assurance  of  accountable  and  efficient 
government  operation. 
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Endnotes 


1.  For  purposes  of  this  article,  the  term  “organizational  structure”  refers  to  a  contractor  and 
its  affiliates,  including  parents  and  subsidiaries. 

2.  This  personal  communication  (letter  dated  July  21,  2010)  from  Mitchell  Vakerics,  policy 
manager  for  The  Coalition  for  Government  Procurement,  replied  to  Amy  Williams,  Office 
of  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics),  Defense 
Procurement  and  Acquisition  Policy  and  Pricing  (Defense  Acquisition  Regulation 
Systems),  issuing  comments  on  the  implementation  of  Section  207  of  the  Weapons 
Systems  Acquisition  Reform  Act  of  2009,  75  Federal  Regulation  20954  (April  22,  2010) 
“Proposed  Rule”  on  Organizational  Conflicts  of  Interest.  Retrieved  from  http://www. 
regulations.gov/#documentDetail;D=DARS=201 0-0045-001 6. 

3.  Because  it  is  only  five  digits  long,  the  Commercial  and  Government  Entity  (CAGE) 

Code  assigned  to  each  contractor  on  registration  is  insufficient  given  the  number  of 
contractors. 

4.  This  personal  communication  (e-mail)  is  courtesy  of  T.  Christian  Williams,  pursuant  to 
his  Freedom  of  Information  Act  (FOIA)  Request  No.  197667  submitted  to  the  General 
Services  Administration. 
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Half-Life  Learning  Curves 
in  the  Defense  Acquisition 
Life  Cycle 


Adedeji  B.  Badiru 


Learning  curves  are  useful  for  assessing  performance 
improvement  due  to  the  positive  impact  of  learning.  In 
recent  years,  the  deleterious  effects  of  forgetting  have 
also  been  recognized.  Workers  experience  forgetting  or 
decline  in  performance  over  time.  Consequently,  contem¬ 
porary  learning  curves  have  attempted  to  incorporate 
forgetting  components  into  learning  curves.  An  area  of 
increasing  interest  is  the  study  of  how  fast  and  how  far 
the  forgetting  impact  can  influence  overall  performance. 
This  article  introduces  the  concept  of  half-life  analysis 
of  learning  curves  using  the  concept  of  growth  and 
decay,  with  particular  emphasis  on  applications  in  the 
defense  acquisition  process.  The  computational  analysis 
of  the  proposed  technique  lends  itself  to  applications 
for  designing  training  and  retraining  programs  for  the 
Defense  Acquisition  Workforce. 
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The  illiterate  of  the  21st  century  will  not  be 

THOSE  WHO  CANNOT  READ  AND  WRITE,  BUT  THOSE 
WHO  CANNOT  LEARN,  UNLEARN,  AND  RELEARN. 

—Alvin  Toffler 


Formal  analysis  of  learning  curves  first  emerged  in  the  mid-1980s 
in  connection  with  the  analysis  of  the  production  of  airplanes  (Wright, 
1936).  Learning  refers  to  the  improved  operational  efficiency  and  cost 
reduction  obtained  from  repetition  of  a  task.  Learning  curves  have 
been  used  for  decades  to  assess  improvement  achieved  over  time  due 
to  the  positive  impacts  of  learning.  Early  analytical  modeling  of  learn¬ 
ing  curves  focused  on  reduction  in  cumulative  average  cost  per  unit  as 
production  level  doubles.  Several  alternate  models  of  learning  curves 
have  been  presented  in  the  literature  of  the  decades.  The  classical  mod¬ 
els  have  been  successfully  applied  to  a  variety  of  problems.  In  recent 
years,  the  deleterious  effects  of  forgetting  have  also  been  recognized.  It 
has  been  shown  that  workers  experience  forgetting  or  decline  in  per¬ 
formance  even  while  they  are  making  progress  along  a  learning  curve. 
Consequently,  contemporary  learning  curves  have  attempted  to  incor¬ 
porate  forgetting  components  into  learning  curves.  An  area  of  increasing 
interest  is  the  study  of  how  fast  and  how  far  the  forgetting  impact  can 
influence  overall  performance. 

This  article  presents  the  concept  of  half-life  analysis  of  learning 
curves  (Badiru,  2010),  using  the  concept  of  growth  and  decay  of  learning 
in  the  acquisition  environment.  Half-life  is  the  amount  of  time  it  takes 
for  a  quantity  to  diminish  to  half  of  its  original  size  through  natural 
processes.  Although  the  common  application  of  half-life  is  in  natural 
sciences,  the  computational  analysis  lends  itself  to  applications  to  learn¬ 
ing  curves.  Several  research  and  application  studies  have  confirmed 
that  human  performance  improves  with  reinforcement  or  frequent  and 
consistent  repetitions.  Badiru  (1992, 1994)  provides  a  computational 
survey  of  learning  curves  as  well  as  industrial  application  to  produc¬ 
tivity  and  performance  analysis.  Reductions  in  operation  processing 
times  achieved  through  learning  curves  can  directly  translate  to  cost 
savings.  In  today's  technology-based  operations,  retention  of  learning 
may  be  threatened  by  fast-paced  shifts  in  operating  requirements.  Thus, 
those  involved  in  computational  analysis  of  learning  curves  may  find  it 
of  benefit  to  study  the  half-life  properties  of  learning  curves.  Informa- 
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tion  about  the  half-life  can  tell  us  something  about  the  sustainability  of 
learning-induced  performance.  This  is  particularly  useful  for  designing 
training  programs  and  assessing  workers'  performance. 

Concept  of  Growth  and  Decay 

Growth  and  decay  occur  naturally  in  many  processes.  We  often 
speak  of  “twice  as  much"  and  “half  as  much"  as  benchmarks  for  process 
analysis.  In  economic  and  financial  principles,  the  “rule  of  72"  refers  to 
the  length  of  time  required  for  an  investment  to  double  in  value.  These 
common  “double"  or  “half"  concepts  provide  the  motivation  for  exam¬ 
ining  half-life  properties  of  learning  curves.  The  usual  application  of 
half-life  is  in  natural  sciences.  For  example,  in  Physics,  the  half-life  is  a 
measure  of  the  stability  of  a  radioactive  substance.  In  practical  terms, 
the  half-life  of  a  substance  is  the  time  it  takes  for  the  substance  to  decay 
to  half  of  its  initial  size.  The  longer  the  half-life  of  a  substance,  the  more 
stable  it  is.  This  provides  a  good  analogy  for  modeling  learning  curves 
with  the  recognition  of  increasing  performance  or  decreasing  cost  with 
respect  to  the  passage  of  time.  For  purposes  of  this  article,  the  following 
key  definitions  are  provided: 

•  Half-life  of  a  learning  curve  is  the  incremental  production 
level  required  to  reduce  cumulative  average  cost  per  unit  to 
half  of  its  initial  level. 

•  Half-life  of  &  forgetting  curve  is  the  amount  of  time  it  takes 
for  performance  to  decline  to  half  of  its  initial  level. 

Literature  on  Learning  Curves 

Although  an  extensive  collection  exists  of  classical  studies  of  perfor¬ 
mance  improvement  due  to  learning  curves,  only  very  limited  attention 
has  been  paid  to  performance  degradation  due  to  the  impact  of  forgetting. 
Some  of  the  classical  works  on  process  improvement  due  to  learning 
include  Smith  (1989);  Belkaoui  (1976, 1986);  Nanda  (1979);  Pegels  (1976); 
Richardson  (1978);  Towill,  and  Kaloo  (1978);  Womer  (1979, 1981, 1984); 
Womer  and  Gulledge  (1983);  Camm,  Evans,  and  Womer  (1987);  Liao 
(1979);  McIntyre  (1977);  Smunt  (1986);  Sule  (1978);  and  Yelle  (1976, 
1979,  1983).  Only  in  recent  years  has  the  recognition  of  “forgetting" 
curves  begun  to  emerge,  as  can  be  seen  in  more  recent  literature  (Badiru, 
1995;  Jaber  &  Sikstrom,  2004;  Jaber,  Hemant,  &  Darwin,  2003;  Jaber  & 
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Bonney,  2003,  2007;  Jaber  &  Guiffrida,  2008).  The  new  and  emerging 
research  on  the  forgetting  components  of  learning  curves  provides  the 
motivation  for  studying  half-life  properties  of  learning  curves.  Perfor¬ 
mance  decay  can  occur  due  to  several  factors,  including  lack  of  training, 
reduced  retention  of  skills,  lapse  in  performance,  extended  breaks  in 
practice,  and  natural  forgetting. 

The  Acquisition  Learning  Framework 

It  is  a  natural  process  for  people  to  learn,  unlearn,  and  relearn.  Cap¬ 
turing  this  process  in  a  quantitative  framework  is  essential  for  making 
effective  decisions  in  any  operation,  particularly  in  the  defense  acquisi¬ 
tion  environment,  where  human-machine  interfaces  are  common. 

Defense  acquisition  endeavors  often  get  behind  schedule,  exceed 
cost  baselines,  and/or  exhibit  poor  performance.  Many  of  these  problems 
have  their  sources  in  the  human  elements  within  the  acquisition  life 
cycle.  Ward  (2010, 2012),  using  his  FIST  (Fast,  Inexpensive,  Simple,  and 
Tiny)  model,  calls  for  rapid  acquisition  using  the  concept  of  “80%  now  is 
better  than  100%  later.”  This  perfectly  fits  the  learning  curve  approach 
proposed  in  this  article. 

Because  the  degradation  of  learning  does  not  follow  a  linear  path, 
it  is  essential  to  monitor  the  various  stages  of  the  learning,  unlearning, 
and  relearning  processes.  This  article  presents  an  analytical  modeling  of 
the  stage  where  a  learning  profile  has  degraded  to  half  of  its  initial  value. 
This  is  useful  for  predicting  the  magnitude  and  behavior  of  learning  over 
time.  The  article  points  out  that  the  half-life  point  is  of  most  interest  in 
tracking  the  degradation  path  of  learning.  That  half-life  point  can  be  used 
for  acquisition  training  and  retraining  purposes.  With  the  techniques  in 
this  article,  something  similar  to  a  break-even  analysis  of  learning  can 
be  done  because  the  upswing  of  learning  and  the  downswing  of  learning 
conceptually  intercept  at  some  point.  Of  particular  note  in  the  decision 
process  is  whether  that  interception  point  occurs  before  or  after  the 
half-life  point.  For  the  purpose  of  training  in  acquisition  operations,  an 
organization  can  use  the  half-life  computational  technique  to  estimate 
what  fraction  of  training  retention  remains  after  some  point  in  time  and 
what  level  of  retraining  might  be  needed  during  the  acquisition  life  cycle. 
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General  Half-Life  Profile 


Figure  1  shows  a  graphical  representation  of  performance  as  a 
function  of  time  under  the  influence  of  forgetting  (i.e.,  performance 
decay).  Performance  decreases  as  time  progresses.  The  objective  is  to 
determine  when  performance  has  decayed  to  half  of  its  original  level. 
Based  on  the  law  of  radioactive  disintegration,  the  Law  of  Learning 
Decay  is  proposed  here. 

The  rate  of  decay  of  learning  due  to  the  effect  of  forgetting  is  pro¬ 
portional,  at  any  instant ,  to  the  incipient  learning  level. 

The  Law  of  Learning  Decay  is  formulated  mathematically  in  subse¬ 
quent  sections  of  this  article.  A  mathematical  abstraction  of  the  physical 
process  of  learning  and  forgetting  is  formulated  by  considering  the  rate 
of  change  in  performance  (P),  which  is  a  function  of  the  learning  rate  (L). 
While  learning  itself  is  difficult  to  quantify  and  measure,  its  output  and 
performance  can  be  measured  as  a  physical  quantity  of  production.  The 
discrete  process  is  approximated  by  a  continuous  curve. 

FIGURE  1.  REPRESENTATION  OF  SYSTEM  RESPONSE  TIME  WITH 

RESPECT  TO  PASSAGE  OF  TIME 
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Thus,  the  following  mathematical  formulation  emerges:  At  time  t,  a 
certain  level  of  learning!/,  yields  a  certain  level  of  performance  denoted 
as  P.  Denote  this  transformation  as: 


L^P 


The  rate  of  decay  of  P  can  be  written  as: 


d\p] 

dt 


~k[Pl 


where  k  =  decay  coefficient.  This  has  the  general  form  of  an  initial  value 
problem  in  first-order  linear  equations,  and  it  has  the  following  general 
solution: 


P(t)  =  P0e~k‘, 


where  PQ=  initial  level  of  performance.  The  half-life  of  P  is  computed  as 
the  value  of  t  at  which  P  decays  to  half  of  its  original  level.  That  is: 


p({ 1/2) = p«e 


—kUr 


m 

UJ 


P0e 


,-kt 0 


which  is  solved  to  obtain  the  half-life  as: 

tn  =—  ln2. 

1/2  k 


To  illustrate  the  application  of  half-life  computations,  consider  an 
engineering  reactor  that  converts  the  relatively  stable  uranium  238  into 
the  isotope  plutonium  239.  After  15  years,  it  is  determined  that  0.043 
percent  of  the  initial  amount  AQ  of  the  plutonium  has  disintegrated. 
Determining  the  half-life  of  the  isotope  is  the  point  of  interest.  From 
Physics,  the  initial  value  problem  is  stated  as: 


with  P( 0)  =  PQ.  This  has  a  general  solution  of  the  form: 


Pit)  =  P0e~kt 
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If  0.043  percent  of  the  atoms  in  AQ  have  disintegrated,  then  99.957 
percent  of  the  substance  remains.  To  find  k,  we  will  solve: 

aP0=P0e-15* 


where  a  =  remaining  fraction  of  the  substance.  With  a  we  obtain  k  = 
0.000028 67.  Thus,  for  any  time  t,  the  amount  of  the  plutonium  isotope 
remaining  is  represented  as: 


p(t) = /)e-°-00002867* 

This  has  a  general  decay  profile  similar  to  the  plot  of  P(f )  in  Figure  1. 
Computation  can  now  be  done  of  the  half-life  as  corresponding  value  at 
time  t  for  which  P(t )  =  PJ 2.  That  is: 


-0.00002867/ 


which  yields  t  (half-life)  value  of 24,180  years.  With  this  general  knowl¬ 
edge  of  the  half-life,  several  computational  analyses  can  be  done  to 
predict  the  behavior  and  magnitude  of  the  substance  over  time.  As 
another  example,  consider  a  radioactive  nuclide,  which  has  a  half-life  of 
30  years.  Suppose  the  interest  lies  in  computing  the  fraction  of  an  ini¬ 
tially  pure  sample  of  this  nuclide  that  will  remain  undecayed  at  the  end 
of  a  time  period,  say  90  years.  From  the  equation  of  half-life,  the  solution 
for  k  can  be  deduced: 

p 

_0_  _  p  g — half -Ufe 

2  0 

,  ln2 

k  = - 

half-life 


Which  gives  k  =  0.0231049.  Now,  we  can  use  this  value  of  k  to  compute: 

=  ^-(0.023 1049)(90)  _  Q  125 

P 
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Similarly,  let  us  consider  a  radioactive  isotope  with  a  half-life  of  140 
days.  The  number  of  days  it  would  take  for  the  sample  to  decay  to  one- 
seventh  of  its  initial  magnitude  can  be  computed: 

p 

_0_  _  p  e~kt  half -life 

2  0 

,  ln2 

ri  = - 

half-life 


Which  yields  k  =  0.004951  and  results  in: 

P  =  l/7P0 

—  P=  P  e~kt 

7  ro 

ln7  , 

t  = - =  393  days 

k 

For  learning  curves,  similar  computational  analysis  can  be  per¬ 
formed  to  assess  the  forgetting-induced  properties  of  the  curves.  Thus, 
a  comparative  analysis  of  the  different  models  can  be  conducted. 
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Half-Life  Application  to  Learning  Curves 

Wright  (1936)  documented  the  “80  percent  learning”  effect,  which 
indicates  that  a  given  operation  is  subject  to  a  20  percent  productivity 
improvement  each  time  the  activity  level  or  production  volume  doubles. 
The  proposed  half-life  approach  is  the  antithesis  of  the  double-level 
milestone.  Some  of  the  classical  learning  curve  models  are: 

•  Log -linear  model 

•  S-curve  model 

•  Stanford-B  model 

•  De  Jong's  learning  formula 

•  Levy’s  adaptation  function  (Levy,  1965) 

•  Glover’s  learning  formula  ( Glover ;  1966) 

•  Pegels’  exponential  function  (Pegels,  1976) 

•  Knecht’s  upturn  model  ( Knecht ,  1974) 

•  Yelle’s  product  model 

The  basic  log-linear  model  is  the  most  popular  learning  curve  model. 
It  expresses  a  dependent  variable  (e.g.,  production  cost)  in  terms  of  some 
independent  variable  (e.g.,  cumulative  production).  The  model  states  that 
the  improvement  in  productivity  is  constant  (i.e.,  it  has  a  constant  slope) 
as  output  increases.  That  is: 

C(x)  =  Clx~b 
Where: 

C(x)  =  cumulative  average  cost  of  producing  x  units 
C1  =  cost  of  the  first  unit 
x  =  cumulative  production  unit 
b  =  learning  curve  exponent 
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Notice  that  the  expression  for  C(x)  is  practical  only  for  x  >  0.  This 
makes  sense  because  learning  effect  cannot  realistically  kick  in  until  at 
least  one  unit  ( x  >  1)  has  been  produced.  For  the  standard  log-linear  model, 
the  expression  for  the  learning  rate,  p,  is  derived  by  considering  two  pro¬ 
duction  levels  where  one  level  is  double  the  other.  For  example,  given  the 
two  levels  x±  and  x2 (where  x2  =  2x1 ),  the  following  expressions  emerge: 

C(x1)  =  C1(x1U 

C(x2)  =  C1(2x1U 

The  percent  productivity  gain,p,  is  then  computed  as: 

C(x2)  C1(2x,)-A  2_» 

c^)  cx(xxyb 

The  performance  curve,  P(t), shown  earlier  in  Figure  1  can  now  be 
defined  as  the  reciprocal  of  the  average  cost  curve,  C(x),  and  as  a  function 
of  production  level,  x.  Thus,  we  have 

1 

coo' 

The  application  of  half-life  analysis  to  learning  curves  can  help 
address  questions  such  as: 

•  How  fast  and  how  far  can  system  performance  be  improved? 

•  What  are  the  limitations  to  system  performance 
improvement? 

•  How  resilient  is  a  system  to  shocks  and  interruptions  to  its 
operation? 

•  Are  the  performance  goals  that  are  set  for  the  system 
achievable? 
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Derivation  of  Half-Life  of  the  Log-Linear  Learning  Curve 

Figure  2  shows  a  pictorial  representation  of  the  basic  log-linear 
model,  with  the  half-life  point  indicated  as  x1/Q.  The  half-life  of  the  log- 

1/  z 

linear  model  is  computed  as  follows: 


CQ  =  Initial  performance  level 


C.  /0  =  Performance  level  at  half-life 

1/  z 

Co  —  CjXq  and  C1/2  —  C,  x1/2 


But  C1/2  ~  ^  C0 


Therefore,  Cxxm 


-Cxx( 


-b 


which  leads  to  x1/2 


Which,  by  taking  the  (-1/6) th  exponent  of  both  sides,  simplifies  to  yield 
the  following  expression  as  the  general  expression  for  the  standard  log- 
linear  learning  curve  model, 


Xl/2  ~ 


(  ^  \ 


v^y 


x0,  x0>l 


where  x  is  the  half-life  and  xQ  is  the  initial  point  of  operation;  X  is  then 
referred  to  as  the  First-Order  Half-Life. 


The  Second- Order  Half-Life  is  computed  as  the  time  corresponding 
to  half  of  the  preceding  half.  That  is: 


C'x~b  =  —  Cx~b 

**1/2(2)  ^  M**0  ’ 


which  simplifies  to  yield: 

2 


*1/2(2) 


v^y 


x0. 


Similarly,  the  Third-Order  Half-Life  is  derived  to  obtain: 


3 

fin 


x 


1/2(3) 


v^y 
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In  general,  the  kth- Order  Half-Life  for  the  log-linear  model  is  repre¬ 
sented  as: 


k 


FIGURE  2.  GENERAL  PROFILE  OF  THE  BASIC  LEARNING  CURVE 
MODEL 


c, 


Computational  Examples 

Figure  3  shows  a  comparison  of  learning  curve  profiles  of  the  log- 
linear  model  with  b  =  0.75  and  b  =  0.3032  respectively.  The  graphical 
profiles  reveal  the  characteristics  of  learning,  which  can  dictate  the 
half-life  behavior  of  the  overall  learning  process.  Knowing  the  point 
where  the  half-life  of  each  curve  occurs  can  be  very  useful  in  assessing 
learning  retention  for  the  purpose  of  designing  training  programs  or 
designing  work. 

For  C(pc )  =  250x  ° 75,  the  First-Order  Half-Life  is  computed  as: 


If  the  above  expression  is  evaluated  for  xQ=  2,  the  first-order  half-life 
yields  x1/2  =  5.0397,  which  indicates  a  fast  drop  in  the  value  of  C(x).The 
specific  case  ofxQ  =  2  shows  C(2)  =  148.6509  corresponding  to  a  half-life 
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of 5.0397.  Note  that  C(5.0397)  =  74.7674,  which  is  about  half  of 148.6509. 
The  conclusion  from  this  analysis  is  that  if  we  are  operating  at  the  point 
x  =  2,  we  can  expect  this  particular  curve  to  reach  its  half-life  decline 
point  at  x  =  5. 

For  C(x )  =  240. 03x  ° 3032  ,  the  First-Order  Half-Life  is  computed  as: 


r  1  0.3032 


If  we  evaluate  the  above  function  for  xQ  =  2,  the  First  Order  Half-Life 
yields  x  =  19.6731.  This  does  not  represent  as  precipitous  a  drop  as  the 
other  curve.  The  half-life  analysis  can  be  applied  to  learning  curves  to 
determine  when  each  cost  element  of  interest  will  decrease  to  half  of 
its  starting  value.  This  information  can  be  useful  for  product  pricing 
purposes,  particularly  for  technology  products  that  are  subject  to  rapid 
price  reductions  due  to  declining  product  cost.  Several  models  and  varia¬ 
tions  of  learning  curves  have  been  reported  in  the  literature  (Badiru, 
1992;  Jaber  &  Guiffrida,  2008).  Models  are  developed  through  one  of  the 
following  approaches: 

1.  Conceptual  models 

2.  Theoretical  models 

3.  Observational  models 

4.  Experimental  models 

5.  Empirical  models 

FIGURE  3.  COMPARISON  OF  LOG-LINEAR  CURVES  FOR  b  =  -0.75 

AND  b  =  -0.3032 
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Half-Life  Derivations  for  Classical  Learning  Models 

The  S-Curve  model.  The  S-Curve  (Towill  &  Cherrington,  1994) 
is  based  on  an  assumption  of  a  gradual  start-up.  The  function  has  the 
shape  of  the  cumulative  normal  distribution  function  for  the  start-up 
curve  and  the  shape  of  an  operating  characteristics  function  for  the 
learning  curve.  The  gradual  start-up  is  based  on  the  fact  that  the  early 
stages  of  production  are  typically  in  a  transient  state  with  changes  in 
tooling,  methods,  materials,  design,  and  even  changes  in  the  workforce. 
The  basic  form  of  the  S-Curve  function  is: 

C(x)  =  Cl+M(x  +  B)~b 

MC(jc)  =  Cj  [M  +  (1  -  M)(x  +  B)~b  ] 

Where: 


C(x)  =  learning  curve  expression 
b  =  learning  curve  exponent 
M(x)  =  marginal  cost  expression 


C  =  cost  of  first  unit 

M  =  incompressibility  factor  (a  constant) 

B  =  equivalent  experience  units  (a  constant). 


Assumptions  about  at  least  three  out  of  the  four  parameters  (M,  B, 
Cx,  and  b)  are  needed  to  solve  for  the  fourth  one.  Using  the  C(x)  expres¬ 
sion  and  derivation  procedure  outlined  earlier  for  the  log-linear  model, 
the  half-life  equation  for  the  S-Curve  learning  model  is  derived  to  be: 


XV2  —  (1  /  2) 


-1  !b 


M(x0+Byb-Cx 

M 


-Mb 


B 


Where: 


x1/2  =  half-life  expression  for  the  S-Curve  Learning  Model 

xQ  =  initial  point  of  evaluation  of  performance  on  the  learning  curve 
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In  terms  of  practical  application  of  the  S-Curve,  consider  when  a 
worker  begins  learning  a  new  task.  The  individual  is  slow  initially  at 
the  tail  end  of  the  S-Curve.  But  the  rate  of  learning  increases  as  time 
goes  on,  with  additional  repetitions.  This  helps  the  worker  to  climb  the 
steep-slope  segment  of  the  S-Curve  very  rapidly.  At  the  top  of  the  slope, 
the  worker  is  classified  as  being  proficient  with  the  learned  task.  From 
then  on,  even  if  the  worker  puts  much  effort  into  improving  upon  the  task, 
the  resultant  learning  will  not  be  proportional  to  the  effort  expended.  The 
top  end  of  the  S-Curve  is  often  called  the  slope  of  diminishing  returns.  At 
the  top  of  the  S-Curve,  workers  succumb  to  the  effects  of  forgetting  and 
other  performance-impeding  factors.  As  the  work  environment  contin¬ 
ues  to  change,  a  worker's  level  of  skill  and  expertise  can  become  obsolete. 
This  is  an  excellent  reason  for  the  application  of  half-life  computations. 
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The  Stanford-B  model.  An  early  form  of  learning  curve  is  the 
Stanford-B  model,  which  is  represented  as: 

UC(x)  =  Cl(x  +  B)~b 

Where: 

UC(x)=  direct  cost  of  producing  the  xthunit 
b  =  learning  curve  exponent 
C  =  cost  of  the  first  unit  when  B  =  0; 

B  =  slope  of  the  asymptote  for  the  curve; 

B  =  constant  (1  <  B  <  10).  This  is  equivalent  units  of  previous  experience  at 
the  start  of  the  process,  which  represents  the  number  of  units  produced 
prior  to  first  unit  acceptance.  It  is  noted  that  when  B  =  0 ,  the  Stanford-B 
model  reduces  to  the  conventional  log-linear  model.  Figure  4  shows  the 
profile  of  the  Stanford-B  model  with  B  =  4.2  and  b  =  -0.75.  The  general 
expression  for  the  half-life  of  the  Stanford-B  model  is  derived  to  be: 

x1/2  =(1/2  ym(x0+B)-B 

Where: 

x1  /0  =  half-life  expression  for  the  Stanford-B  Learning  Model 

1/ 

xQ  =  initial  point  of  evaluation  of  performance  on  the  learning  curve 

FIGURE  4.  STANFORD-B  MODEL  WITH  PARAMETERS  B  =  4.2  AND 
b  -  -0.75 
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Multifactor  Half-Life  Derivation 

Badiru  (1994)  presents  applications  of  learning  and  forgetting  curves 
to  productivity  and  performance  analysis.  One  example  presented  used 
production  data  to  develop  a  predictive  model  of  production  through¬ 
put.  Two  data  replicates  are  used  for  each  of  10  selected  combinations 
of  cost  and  time  values.  Observations  were  recorded  for  the  number  of 
units  representing  double  production  levels.  The  resulting  model  has 
the  functional  form  below  and  the  graphical  profile  shown  in  Figure  5. 


C(x)  =  298.88x1“°'31x2“013 


Where: 

C(x)  =  cumulative  production  volume 
x1  =  cumulative  units  of  Factor  1 
x2  =  cumulative  units  of  Factor  2 
bx  -  First  learning  curve  exponent  =  -0.31 
b2  =  Second  learning  curve  exponent  =  -0.13 

A  general  form  of  the  modeled  multifactor  learning  curve  model  is: 


C(x)  =  c,x;h'x;h-- 


and  the  half-life  expression  for  the  multifactor  learning  curve  was 
derived  to  be: 


X2(i/2)  =  (1  /  2)-I/6= 


x1(I/2)  =  (1  /  2)~Vb' 
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Where: 

x.(1/2)=  half-life  component  due  to  Factor  i  ( i  =  1,  2) 

x.(0)=  initial  point  of  Factor  i  (i  =  1, 2)  along  the  multifactor  learning  curve 

Knowledge  of  the  value  of  one  factor  is  needed  to  evaluate  the  other 
factor.  Just  as  in  the  case  of  single-factor  models,  the  half-life  analysis 
of  the  multifactor  model  can  be  used  to  predict  when  the  performance 
metric  will  reach  half  of  a  starting  value. 


FIGURE  5.  BIVARIATE  MODEL  OF  LEARNING  CURVE 
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Incorporation  of  Forgetting  Functions  into  Learning  Curves 

Several  factors  can  influence  learning  rate  in  practice.  A  better 
understanding  of  the  profiles  of  learning  curves  can  help  in  the  devel¬ 
opment  of  forgetting  intervention  programs  and  for  the  assessment  of 
the  sustainability  of  learning.  For  example,  shifting  from  learning  one 
operational  process  to  another  can  influence  the  half-life  profile  of  the 
original  learning  curve.  Important  questions  that  half-life  analysis  can 
address  include  the  following: 

1.  What  factors  influence  learning  retention  and  for  how  long? 

2.  What  factors  foster  forgetting  and  at  what  rate? 

3.  What  joint  effects  exist  to  determine  the  overall  learning 
profile  for  worker  performance  and  productivity? 

4.  What  is  the  profile  of  and  rate  of  decline  of  the  forgetting 
curve? 

The  issues  related  to  the  impact  of  forgetting  in  performance  and 
productivity  analysis  are  brought  to  the  forefront  by  Badiru  (1994, 1995) 
and  all  the  references  therein.  Retention  rate  and  retention  capacity  of 
different  workers  will  determine  the  nature  of  the  forgetting  function  to 
be  modeled  for  the  workers.  Whenever  interruption  occurs  in  the  learn¬ 
ing  process,  as  in  scheduled  breaks  (Anderlohr,  1969),  it  results  in  some 
forgetting.  The  resulting  drop  in  performance  rate  depends  on  the  initial 
level  of  performance  and  the  length  of  the  interruption.  The  following 
three  potential  cases  illustrate  how  forgetting  may  occur: 

Case  1:  Forgetting  may  occur  continuously  throughout  the  learning 
process. 

Case  2:  Forgetting  may  occur  discretely  over  distinct  bounded  time 
intervals. 

Case  3:  Forgetting  may  occur  over  intermittent  and/or  random  time 
intervals  where  the  time  of  occurrence  and  duration  of  forgetting  are 
described  by  some  probability  distribution. 
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Any  operation  that  is  subject  to  interruption  in  the  learning  process 
is  susceptible  to  the  impact  of  forgetting.  Sule  (1978)  postulated  that  the 
forgetting  model  can  be  represented  as: 

~bf 

yf=xfrf  > 


where: 

yf  =  number  of  units  that  could  be  produced  on  rth  day  in  the  presence  of 
forgetting. 

x  =  equivalent  production  on  first  day  of  the  forgetting  curve. 
r  =  cumulative  number  of  days  in  the  forgetting  cycle. 
b  =  forgetting  rate. 

The  forgetting  function  has  the  same  basic  form  as  the  standard 
learning  curve  model,  except  that  the  forgetting  rate  will  be  negative, 
indicating  a  decay  process.  Figure  6  shows  some  of  the  possible  profiles 
of  the  forgetting  curve.  Profile  (a)  shows  a  case  where  forgetting  occurs 
rapidly  along  a  convex  curve.  Profile  (b)  shows  a  case  where  forgetting 
occurs  more  slowly  along  a  concave  curve.  Profile  (c)  shows  a  case  where 
the  rate  of  forgetting  shifts  from  convex  to  concave  along  an  S-Curve. 

FIGURE  6.  ALTERNATE  PROFILES  DECLINING  IMPACT  OF 
FORGETTING 


TIME  AXIS 

The  profile  of  the  forgetting  curve  and  its  mode  of  occurrence  can 
influence  the  half-life  measure.  This  is  further  evidence  that  the  com¬ 
putation  of  half-life  can  help  distinguish  between  learning  curves, 
particularly  if  a  forgetting  component  is  involved.  The  combination  of 
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the  learning  and  forgetting  functions  presents  a  more  realistic  picture 
of  what  actually  occurs  in  a  learning  process.  The  combination  is  not 
necessarily  as  simple  as  resolving  two  curves  to  obtain  a  resultant  curve. 
The  resolution  may  particularly  be  complex  in  the  case  of  intermittent 
periods  of  forgetting.  Figure  7  shows  representations  of  periods  where 
forgetting  occurs  and  the  resultant  learn-forget  profile. 


FIGURE  7.  RESOLUTION  OF  LEARN-FORGET  PERFORMANCE 
CURVES 


Applications  to  Training  and  Worker  Effectiveness  Analysis 

Learning  curves  are  traditionally  used  for  diagnostic  and  planning 
purposes  in  installed  operations.  The  premise  of  this  article  is  that 
learning  curve  analysis,  learn-forget  modeling,  and  half-life  analysis 
can  be  used  proactively  to  design  or  enhance  training  programs,  thereby 
improving  worker  effectiveness.  Training  is  a  capital-intensive  overhead 
cost  that  is  often  difficult  to  justify  in  terms  of  revenue  production.  There 
are  two  aspects  of  justifying  training  programs:  effectiveness  and  effi¬ 
ciency  of  the  training  program.  Effectiveness  refers  to  the  benefits  an 
organization  derives  from  training  the  workforce  to  meet  organizational 
objectives.  Efficiency  refers  to  the  process  of  determining  the  resources 
required  for  the  training  versus  the  expected  output.  In  this  process,  it 
is  essential  to  provide  the  resources  required  at  the  right  time,  in  the 
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right  form,  and  in  the  right  quantity.  An  understanding  of  the  half-life 
characteristics  of  the  learning  process  can  make  the  resources  allocation 
process  more  effective. 

In  practice,  there  is  a  lack  of  a  structured  approach  to  ensuring 
training  effectiveness  and  efficiency.  Sawhney,  Badiru,  and  Niranjan 
(2004)  present  a  structured  model  training.  The  model  is  adapted  here  to 
show  where  learning  curve  analysis  may  be  important  and  how  half-life 
analysis  can  be  incorporated.  Figure  8  shows  the  streamlined  training 
process  incorporating  learning  curve  analysis,  forgetting  analysis,  and 
half-life  analysis.  The  first  phase  is  to  assess  the  alignment  of  the  train¬ 
ing  program  to  the  organizational  strategic  goals  in  light  of  the  learning 
curve  impact.  Phase  2  involves  specific  design  of  the  training  program 
with  recognition  of  the  learn-forget  phenomenon.  Phase  3  addresses 
training  implementation  with  respect  to  the  limit  of  the  learning  effect, 
half-life  properties  of  learning,  and  the  limit  of  retention.  Phase  4  final¬ 
izes  the  process  with  training  enhancement  activities.  This  can  involve 
resource  realignment,  output  evaluation,  and  risk  mitigation  for  the 
subsequent  rounds. 

FIGURE  8.  INCORPORATION  OF  LEARNING,  FORGETTING,  AND 

HALF-LIFE  ANALYSIS  INTO  TRAINING  PROCESS 

PHASE  1  PHASE  2  PHASE  3  PHASE  4 


Analysis  Analysis  Analysis  Assessment 


Conclusions 


Degradation  of  performance  occurs  naturally  either  due  to  internal 
processes  or  externally  imposed  events,  such  as  extended  production 
breaks.  For  productivity  assessment  purposes,  it  may  be  of  benefit  to 
determine  the  length  of  time  it  takes  a  production  metric  to  decay  to  half 
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of  its  original  magnitude.  For  example,  for  career  planning  strategy,  one 
may  be  interested  in  how  long  it  takes  for  skills  sets  to  degrade  by  half 
in  relation  to  current  technological  needs  of  the  workplace.  The  half-life 
phenomenon  maybe  due  to  intrinsic  factors,  such  as  forgetting,  or  due  to 
external  factors,  such  as  a  shift  in  labor  requirements.  Half-life  analy¬ 
sis  can  have  application  in  intervention  programs  designed  to  achieve 
reinforcement  of  learning.  It  can  also  have  application  for  assessing  the 
sustainability  of  skills  acquired  through  training  programs.  Further 
research  on  the  theory  of  half-life  of  learning  curves  should  be  directed 
to  topics  such  as  the  following: 

•  Half-Life  Interpretations 

•  Training  and  Learning  Reinforcement  Program 

•  Forgetting  Intervention  and  Sustainability  Programs 

In  addition  to  the  predictive  benefits  of  half-life  expressions,  they 
also  reveal  the  ad  hoc  nature  of  some  of  the  classical  learning  curve 
models  that  have  been  presented  in  the  literature.  The  author  recom¬ 
mends  that  future  efforts  to  develop  learning  curve  models  should  also 
attempt  to  develop  the  corresponding  half-life  expressions  to  provide 
full  operating  characteristics  of  the  models.  Readers  are  encouraged  to 
explore  half-life  analysis  of  other  learning  curve  models  not  covered  in 
this  article. 

Author  Biography 

Professor  Adedeji  Badiru  is  head  of  Systems 
and  Engineering  Management  at  the  Air  Force 
Institute  of  Technology.  He  is  a  registered 
professional  engineer,  a  certified  Project 
Management  Professional  (PMP),  and  a  Fel¬ 
low  of  the  Institute  of  Industrial  Engineers. 
He  has  BS  and  MS  degrees  in  Industrial 
Engineering  (IE),  an  MS  in  Mathematics 
from  Tennessee  Technological  University, 
and  a  PhD  in  IE  from  the  University  of  Central 
Florida.  He  has  authored  several  books  and 
journal  articles. 

(E-mail  address:  adedeji.badiru@afit.edu) 


Defense  ARJ,  July  2012,  Vol.  19  No.  3 : 283-308 


306 


A  Publication  of  the  Defense  Acquisition  University 


http :  //w  w  w.  dau.  mil 


References 


Anderlohr,  G.  (1969).  What  production  breaks  cost.  Industrial  Engineering,  7(9),  34-36. 

Badiru,  A.  B.  (1992).  Computational  survey  of  univariate  and  multivariate  learning  curve 
models.  IEEE  Transactions  on  Engineering  Management,  39(2),  176-188. 

Badiru,  A.  B.  (1994).  Multifactor  learning  and  forgetting  models  for  productivity  and 

performance  analysis.  International  Journal  of  Human  Factors  in  Manufacturing,  4(1), 
37-54. 

Badiru,  A.  B.  (1995).  Multivariate  analysis  of  the  effect  of  learning  and  forgetting  on  product 
quality.  International  Journal  of  Production  Research,  33(3),  777-794. 

Badiru,  A.  B.  (2010).  Half-life  of  learning  curves  for  information  technology  project 
management.  International  Journal  of  IT  Project  Management,  7(3),  28-45. 

Belkaoui,  A.  (1976),  Costing  through  learning.  Cost  and  Management,  50(3),  36-40. 

Belkaoui,  A.  (1986).  The  learning  curve.  Westport,  CT:  Quorum  Books. 

Camm,  J.  D.,  Evans,  J.  R.,  &  Womer,  N.  K.  (1987).  The  unit  learning  curve  approximation  of  total 
cost.  Computers  and  Industrial  Engineering,  12(3),  205-213. 

Glover,  J.  H.,  (1966).  Manufacturing  progress  functions:  An  alternative  model  and  its 

comparison  with  existing  functions.  International  Journal  of  Production  Research,  4(4), 
279-300. 

Jaber,  M.  Y.,  &  Bonney,  M.  (1996).  Production  breaks  and  the  learning  curve:  The  forgetting 
phenomena.  Applied  Mathematical  Modelling,  20(2),  162-169. 

Jaber,  M.  Y.,  &  Bonney,  M.  (2003).  Lot  sizing  with  learning  and  forgetting  in  setups  and  in 
product  quality.  International  Journal  of  Production  Economics,  83(1),  95-111. 

Jaber,  M.  Y.,  &  Bonney,  M.  (2007).  Economic  Manufacture  Quantity  (EMQ)  model  with  lot  size 
dependent  learning  and  forgetting  rates.  International  Journal  of  Production  Economics, 
708(1-2),  359-367. 

Jaber,  M.Y.,  &  Guiffrida,  A.  L.  (2008).  Learning  curves  for  imperfect  production  processes 
with  reworks  and  process  restoration  interruptions.  European  Journal  of  Operational 
Research,  789(1),  93-104. 

Jaber,  M.  Y.,  Hemant,  K.  V.,  &  Darwin  D.  J.  (2003).  Countering  forgetting  through  training  and 
deployment.  International  Journal  of  Production  Economics,  85(1),  33-46. 

Jaber,  M.  Y.,  &  Sikstrom,  S.  (2004).  A  numerical  comparison  of  three  potential  learning  and 
forgetting  models.  International  Journal  of  Production  Economics,  92(3),  281-294. 

Knecht,  G.  R.  (1974).  Costing,  technological  growth,  and  generalized  learning  curves. 
Operations  Research  Quarterly,  25(3),  487-491. 

Levy,  F.  K.  (1965).  Adaptation  in  the  production  process.  Management  Science,  77(6),  B136- 
B154. 

Liao,  W.  M.  (1979).  Effects  of  learning  on  resource  allocation  decisions.  Decision  Sciences, 

70(1),  116-125. 

McIntyre,  E.  V.  (1977).  Cost-volume-profit  analysis  adjusted  for  learning.  Management  Science, 
24(2),  149-160. 

Nanda,  R.  (1979).  Using  learning  curves  in  integration  of  production  resources.  Proceedings  of 
the  1979  HE  Fall  Conference,  376-380. 

Pegels,  C.  C.  (1976).  Start  up  or  learning  curves— Some  new  approaches.  Decision  Sciences, 
7(4),  705-713. 

Richardson,  W.  J.  (1978).  Use  of  learning  curves  to  set  goals  and  monitor  progress  in  cost 
reduction  programs.  Proceedings  of  1978  HE  Spring  Conference,  235-239. 


307 


Defense ARJ,  July  2012,  Vol.  19 No.  3 : 283-308 


Half-Life  Learning  Curves  in  the  Defense  Acquisition  Life  Cycle 


Sawhney,  R.,  Badiru  A.  B.,  &  Niranjan,  A.  (2004).  A  model  for  integrating  and  managing 

resources  for  technical  training  programs.  In  Y.  A.  Hosni  and  T.  M.  Khalil  (Eds.),  Internet 
Economy:  Opportunities  and  Challenges  for  Developed  and  Developing  Regions  of  the 
World,  pp.  337-351.  Boston,  MA:  Elsevier  Ltd. 

Smith,  J.  (1989).  Learning  curve  for  cost  control.  Norcross,  GA:  Industrial  Engineering  and 
Management  Press. 

Smunt,  T.  L.  (1986).  A  comparison  of  learning  curve  analysis  and  moving  average  ratio  analysis 
for  detailed  operational  planning.  Decision  Sciences ,  77(4),  475-495. 

Sule,  D.  R.  (1978).  The  effect  of  alternate  periods  of  learning  and  forgetting  on  economic 
manufacturing  quantity.  AIIE  Transactions,  70(3),  338-343. 

Towill,  D.  R.,  &  Cherrington,  J.  E.  (1994).  Learning  curve  models  for  predicting  the  performance 
of  advanced  manufacturing  technology.  International  Journal  of  Advanced  Manufacturing 
Technology,  9(3),  195-203. 

Towill,  D.  R.,  &  Kaloo,  U.  (1978).  Productivity  drift  in  extended  learning  curves.  Omega,  6(4), 
295-304. 

Ward,  D.  (2010).  Faster,  better,  cheaper  revisited:  Program  management  lessons  from  NASA. 
Defense  AT&L  magazine,  213(2),  48-52. 

Ward,  D.  (2012).  Faster,  better,  cheaper:  Why  not  pick  all  three?  National  Defense.  Retrieved 
from  http://www.nationaldefensemagazine.org/archive/2012/April/Pages/Faster, Better, 
CheaperWhyNotPickAIIThree.aspx?PF=l 

Womer,  N.  K.  (1979).  Learning  Curves,  production  rate,  and  program  costs.  Management 
Science,  25(4),  312-219. 

Womer,  N.  K.  (1981).  Some  propositions  on  cost  functions.  Southern  Economic  Journal,  47(4), 
1111-1119. 

Womer,  N.  K.  (1984).  Estimating  learning  curves  from  aggregate  monthly  data.  Management 
Science,  30(8),  982-992. 

Womer,  N.  K.,  &  Gulledge,  T.  R.  (1983).  A  dynamic  cost  function  for  an  airframe  production 
program.  Engineering  Costs  and  Production  Economics,  7(3),  213-227. 

Wright,  T.  P.  (1936).  Factors  affecting  the  cost  of  airplanes.  Journal  of  Aeronautical  Science, 
3(2),  122-128. 

Yelle,  L.  E.  (1976).  Estimating  learning  curves  for  potential  products.  Industrial  Marketing 
Management,  5(2/3),  147-154. 

Yelle,  L.  E.  (1979).  The  learning  curve:  Historical  review  and  comprehensive  survey.  Decision 
Sciences,  70(2),  302-328. 

Yelle,  L.  E.  (1983).  Adding  life  cycles  to  learning  curves.  Long  Range  Planning,  76(6),  82-87. 


Defense  ARJ,  July  2012,  Vol.  19  No.  3 : 283-308 


308 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


Keywords:  Decision  Cost  Model  ( DCM ),  Decision 
Tree  Node,  Cost-Overrun  Percentage,  Probability, 
Pearson-Tukey  Method 
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As  U.S.  Government  facilities  age  and  new  facilities 
are  constructed,  the  need  to  hire  contractors  for  an 
increasing  number  of  government  construction  projects 
is  imperative.  The  current  government  technical  evalu¬ 
ation  for  contractor  selection  is  less  than  optimal.  This 
article  introduces  an  alternative  technical  evaluation 
methodology  to  the  current  government  contractor 
selection  process:  a  Decision  Cost  Model  (DCM)  that 
can  be  applied  to  ensure  cost-efficient  contractors  are 
selected  in  awarding  construction  contracts.  Applying 
the  DCM  ensures  contractors  with  the  lowest  expected 
total  cost  are  recommended  for  project  awards.  Also 
presented  are  ways  DCM  can  be  applied  to  increase 
efficiency  in  the  selection  process  for  future  government 
construction  projects,  while  simultaneously  meeting 
taxpayers’  expectations  of  receiving  maximum  value 
for  their  tax  dollars. 
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Applying  decision  cost  analysis  provides  the  U.S.  Government  an 
alternative  to  the  existing  process  for  selecting  construction  contrac¬ 
tors.  The  Decision  Cost  Model  (DCM)  proposed  in  this  article  evaluates 
each  prospective  contractor  against  computed  cost  factors  and  uses  the 
contractor's  cost  estimates  to  compute  the  total  expected  cost  for  con¬ 
struction  projects.  The  DCM  can  be  used  with  any  number  of  contractors 
and  with  any  number  of  construction  division  categories. 

The  assessment  of  cost  overruns  is  based  on  an  evaluation  of  historic 
data  from  recent/similar  projects  undertaken  by  government  contractors. 
The  model  considers  five  recent/similar  projects  for  each  contractor.  In 
general,  the  more  recent/similar  projects  used  in  the  modeling  analysis, 
the  better  the  modeling  of  total  expected  cost.  In  the  event  a  contractor 
does  not  have  similar  project  data,  the  contractor  is  omitted  from  the 
contractor  selection  pool.  The  DCM  considers  cost  factors  for  specific 
construction  division  estimates  and  cost-overrun  percentages.  Applying 
the  DCM  requires  the  evaluator  or  project  managers  to  collect  historical 
cost  data  to  compute  the  division  cost  factors.  In  many  cases,  historical 
cost  data  available  for  the  project  cost  analysis  may  be  limited.  However, 
with  expert  judgment  and  careful  evaluation  of  each  division  cost  fac¬ 
tor,  the  program  manager  (PM)  can  ensure  each  contractor  is  evaluated 
equitably.  Note  that  just  about  every  division  has  the  potential  of  cost 
overruns.  Those  without  a  cost  overrun  indicate  the  contractor  has  cost 
control  over  the  underlying  construction  division(s),  and  this  control  of 
costs  will  not  negatively  impact  the  total  expected  cost  of  the  project. 

For  purposes  of  this  study,  the  DCM  method  is  applied  using  the 
example  of  three  contractors  and  three  cost  factors  and  their  computed 
division  cost  overrun  percentages.  The  DCM  application  compares 
division  costs  for  electrical,  structural,  and  mechanical  contractor 
expenditures.  The  cost  factors  are  modeled  using  historical  data  and 
expert  judgment  combined  with  a  probability  model  to  fit  a  cost-overrun 
percentage  distribution  for  each  cost  factor.  The  Pearson-Tukey  method 
is  used  to  apply  cost-overrun  probabilities  to  chance  nodes  in  a  three- 
outcome  decision  tree  (Clemen,  2001).  The  central  idea  is  to  find  three 
representative  points  in  the  distribution  and  assign  respective  prob¬ 
ability  values  to  each  outcome.  Accordingly,  the  Pearson-Tukey  method 
allows  the  PM  to  pick  the  most  representative  points  and  probability 
values  for  each  cost  factor. 


311 


Defense  ARJ,  July  2012,  Vol.  19  No.  3 : 309-344 


Decision  Cost  Model  for  Contractor  Selection 


The  intent  of  this  article  is  to  demonstrate  the  development  of  the 
DCM  decision  method  and  apply  the  method  with  an  example  utilizing 
real-world  data.  The  DCM  method  gives  an  approximation  of  how  divi¬ 
sion  cost-overrun  percentages  impact  the  division  estimate  and  the  total 
expected  cost  for  the  project  (Clemen,  2001).  Hence,  the  DCM  method 
provides  a  novel  way  to  improve  future  government  contractor  selections 
for  awarding  government  construction  projects.  The  DCM  improves  the 
existing  contractor  selection  process  by  adding  an  evaluation  of  potential 
cost-overruns  in  computing  the  total  expected  cost  for  a  project. 

The  next  three  sections  of  the  article  describe  the  current  proj¬ 
ect  award  process  and  how  the  DCM  can  easily  be  inserted  into  the 
current  process;  introduce  a  cost  factor  data  table;  and  provide  a  com¬ 
plete  description  of  the  DCM  and  its  methodology.  Following  the  DCM 
methodology,  the  article  discusses  applying  the  DCM  in  depth,  using 
a  real-world  example  with  historical  cost  data  and  expert  judgment. 
Finally,  the  article  concludes  by  reporting  the  DCM  total  expected  proj¬ 
ect  costs  and  the  author's  recommendation  for  future  use  of  the  DCM. 

Current  Project  Award  Process 

"Best  value  technically  acceptable"  is  a  term  used  by  the  government 
to  select  a  project  contractor  meeting  the  technically  acceptable  criterion 
at  least  cost  (General  Services  Administration,  2005).  If  a  contractor  has 
the  lowest  cost  estimate  and  has  met  the  technically  acceptable  criteria, 
then  the  contractor  is  awarded  the  project  contract.  The  current  govern¬ 
ment  "best  value  technically  acceptable"  criterion  does  not  take  into 
account  the  impact  of  potential  project  cost  overruns  on  the  final  cost 
of  a  government  project.  This  is  a  shortcoming  in  conducting  feasibility 
analysis  for  construction  projects. 

Government  construction  projects  are  projected  years  in  advance 
of  their  purposed  construction  or  operation.  To  understand  the  needs, 
requirements,  and  budget  allocation  for  constructing  facilities,  gov¬ 
ernment  decision  makers  require  more  accurate  feasibility  studies.  In 
general,  a  feasibility  study  contains  a  needs  analysis,  mission  require¬ 
ments,  and  cost  estimate  for  the  project.  Consequently,  the  feasibility 
study  is  instrumental  in  awarding  project  contracts.  The  government 
cost  estimate  contained  in  the  feasibility  study  provides  guidance  in 
the  solicitation  of  contractors  for  the  project.  Generally,  the  contractor 
solicitation  for  a  given  construction  project  will  request  two  forms  of 
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cost  information:  a  project  cost  estimate  for  a  facility  meeting  the  pro¬ 
jected  needs  and  performance  standards,  and  the  projected  contractor's 
historical  cost  and  performance  data  relevant  to  the  project.  The  aim  is 
to  forecast  the  costs  required  to  complete  the  construction  project  in 
accordance  with  the  contract  and  plans.  Construction  project  estimation 
is  a  difficult  and  time-consuming  process.  Engineering  and  contractor 
experience  are  needed  to  complete  a  good  estimate.  The  PM  must  assume 
the  roles  of  both  contractor  and  engineer  to  ensure  sound  contracting 
and  engineering  principles  are  adhered  to  in  the  construction  project. 

The  government  PM  represents  the  taxpayer  and  is  responsible  for 
developing  the  construction  project's  Independent  Government  Estimate 
(IGE).  Because  construction  projects  are  projected  years  in  advance  of 
their  need,  the  government  PM  will  prepare  a  current  year  IGE  for  the 
project.  The  current  year  IGE  is  utilized  to  perform  the  technical  evalu¬ 
ation  and  compare  the  project  cost  estimates  submitted  by  contractors. 
The  current  IGE  represents  the  total  estimated  cost  of  the  project  and 
division  estimated  costs.  The  IGE  is  developed  using RSMeans,  an  indus¬ 
try  standard  for  construction  cost  estimation.  RSMeans  is  a  division  in 
Reed  Construction  Data  that  provides  costs  by  discipline  format,  site 
prep,  mechanical,  and  electrical.  The  division  specializes  in  providing 
material,  labor,  and  building  cost  information  to  the  North  America  con¬ 
struction  industry.  (Note  that  RSMeans  cost  data  are  updated  annually 
and  delivered  in  a  book  or  software  application.) 

The  Construction  Specifications  Institute  (CSI)  is  an  organiza¬ 
tion  that  maintains  and  advances  the  standardization  of  construction 
language,  as  pertains  to  building  specifications.  CSI  Master  Format  is 
an  indexing  system  for  organizing  construction  data  and  construction 
specifications.  For  purposes  of  this  article,  the  CSI  Master  Format  con¬ 
siders  16  divisions  of  construction  costs.  RSMeans  cost  data  are  available 
on  a  software  program  called  CostWorks  and  in  RSMeans  construction 
cost  data  manuals.  RSMeans  is  a  cost  data  source,  which  has  45,000 
separate  cost  line  items  for  all  areas  of  construction.  Each  cost  line  item 
represents  data  collected  to  represent  the  mean  average  of  material, 
labor,  and  equipment.  This  cost  is  gathered  from  30  cities  throughout  the 
United  States.  The  RSMeans  database  is  updated  annually,  and  the  data 
are  adjusted  to  the  area  of  the  country  where  the  construction  is  occur¬ 
ring.  For  the  purpose  of  construction  cost  estimation,  the  RSMeans  data 
come  in  two  formats:  RSMeans  2004,  which  has  a  50-division  format, 
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and  RSMeans  1998,  which  has  a  16-division  format.  Both  formats  have 
the  same  basic  information,  with  RSMeans  2004  separated  into  the  basic 
information,  which  is  further  separated  into  more  specialty  divisions. 

RSMeans  1998  defines  construction  disciplines  by  16  separate  areas 
identified  as  divisions: 

1.  Division  1  is  “General  Requirements.”  General  Requirements  are 
items  such  as  supervisor,  project  manager  costs,  vehicles,  and 
other  general  items  required  for  construction. 

2.  Division  2  is  “Site  Construction.”  Site  Construction  is  dirt 
work,  surveys,  and  the  site  preparation  required  for  building 
construction. 

3.  Division  3  is  “Concrete.”  Due  to  the  expense  involved  in  concrete 
and  the  volatile  market  for  concrete,  this  must  be  identified 
separately. 

4.  Division  4  is  “Masonry.”  This  section  identifies  block  work  in 
basements,  fencing,  and  subfloor  needs. 

5.  Division  5  is  “Metals.”  This  identifies  all  metal  materials  used  for 
construction,  including  siding,  studs,  and  metal  work. 

6.  Division  6  is  “Woods  and  Plastics.”  This  area  identifies  all  doors, 
hardware,  and  special  Panduit®  products  (pertaining  to  tubing, 
panels,  electronic  cables,  etc.). 

7.  Division  7  is  “Thermal  and  Moisture  Protection.”  This  identi¬ 
fies  items  involved  in  insulation,  vapor-barrier  protection,  and 
sealants. 

8.  Division  8  is  “Doors  and  Windows.”  This  includes  any  that  may 
be  required  to  support  the  project. 

9.  Division  9  is  “Finishes.”  Finishes  include  paint,  flooring,  and 
molding. 
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10.  Division  10  is  “Specialties  .”  Specialties  include  alarm  systems 
and  all  other  special  equipment.  This  could  pose  huge  variability 
in  project  costs. 

11.  Division  11  is  “Equipment.”  This  includes  items  located  inside 
the  building,  such  as  safes,  electronics,  and  any  other  types  of 
equipment  (other  than  Division  15  Mechanical)  that  could  be 
considered  a  permanent  item  inside  the  building. 

12.  Division  12  is  “Furnishings.”  This  includes  items  such  as  furni¬ 
ture  for  offices  or  millwork. 

13.  Division  13  is  “Special  Construction.”  This  encompasses  items 
that  may  fall  outside  normal  construction.  This  would  include 
fire  protection  and  special  electronic  needs. 

14.  Division  14  is  “Conveying  Systems.”  This  includes  special 
furnishings. 

15.  Division  15  is  “Mechanical.”  This  includes  heating,  ventilation, 
cooling,  and  special  operations  of  doors  or  ventilation  systems. 

16.  Division  16  is  “Electrical.”  This  addresses  electrical  supplies 
used  for  any  electrical  needs  inside  or  outside  the  building.  It  also 
includes  items  that  may  be  used  to  supply  power  to  the  building. 

Decision  Cost  Model 

The  DCM  proposed  in  this  article  can  be  implemented  at  the  tech¬ 
nical  evaluation  stage  in  selecting  a  contractor  for  a  given  construction 
project.  The  proposed  model  computes  the  total  expected  costs  of  a  con¬ 
struction  project  by  modeling  cost-overrun  percentages  for  each  division 
cost  factor  combined  with  the  division  cost  estimate.  The  DCM  utilizes 
the  same  contractor  historical  data  and  estimates  that  are  used  in  the 
current  contractor  selection  process.  The  key  difference  between  the 
current  contractor  selection  process  and  applying  the  proposed  method 
is  allowing  a  more  detailed  evaluation  of  each  division's  estimate  and  the 
impacts  of  cost-overrun  percentages  computed  from  similar  projects. 
The  DCM  uses  the  existing  estimates  and  cost  factors  to  determine 
which  contractor  offers  the  lowest  expected  construction  project  costs. 
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The  DCM  model  takes  into  account  all  estimated  costs,  cost-over¬ 
run  percentages,  and  PM  expert  judgment  of  cost-overrun  risk.  The 
DCM  uses  the  same  common  RSMeans  format  for  comparing  contrac¬ 
tor's  cost  estimates,  thus  reducing  subjectivity  in  the  selection  process. 
The  DCM  uses  commercially  available  software  to  facilitate  contractor 
selection  for  project  awards.  By  identifying  the  lowest  total  cost  con¬ 
tractor  for  the  project,  the  DCM  provides  expected  value  information 
for  improving  efficiencies  in  allocating  taxpayer  funds  for  government 
construction  projects. 

The  DCM  example  used  in  this  article  compares  three  prospective 
contractors  for  a  real-world  project,  identified  as  Contractors  1-3.  The 
cost  estimate  data  used  for  calculations  are  “total  estimate”  and  “per¬ 
cent  change”  in  division  costs.  Based  on  the  16  potential  cost  divisions 
in  RSMeans,  the  study  assumes  the  PM  has  selected  three  cost-overrun 
divisions  facing  cost-overrun  risk.  The  corresponding  cost-overrun 
factors  are:  cost  factor  1-mechanical,  cost  factor  2-finish,  and  cost  fac¬ 
tor  3-electrical. 

To  apply  the  DCM,  the  PM  must  specify  a  probability  model  for  each 
cost  factor  under  consideration.  The  beta-general  distribution  is  well 
suited  for  cost  analysis  under  uncertainty.  The  beta  distribution  is  par¬ 
simonious  and  flexible  when  applying  expert  judgments.  However,  in 
applying  the  beta  distribution  in  cost  analysis,  the  PM  must  estimate 
best-fit  parameters.  The  approach  taken  in  this  article  is  to  calculate 
these  parameters  by  minimization  of  absolute  difference  of  the  probability 
distribution  estimates  for  various  cost-overrun  percentages.  Accordingly, 
the  following  approach  is  used  to  pick  the  best  parameter  set  for  cost 
factors  in  the  [0,1]  range  Min  SJL,  df  where  dt  =  | fB  (yt)  - 
are  absolute  differences  of  probability  distribution  estimates  for  various 
cost-overrun  percentages,  with  “B”  subscript  denoting  the  beta  distribu¬ 
tion.  The  y,  values  are  cost-overrun  percentage  fractiles,  expressed  as  a 
number  in  the  [0, 1]  range  for  the  factor  in  question.  These  cost-overrun 
percentage  fractiles  and  the  PM-assessed  distribution  fractile,  i.e.,  fB 
combine  actual  data  and  expert  judgment. 

The  objective  is  to  find  beta  distribution  parameters  for  the  lower 
bound  “a”  and  upper  bound  “b”  denoted  by  This  ensures  the 

modeled  beta  distribution  fB  is  the  best  approximation  for  various 
cost-overrun  factors.  The  probability  model  combines  historical  data  and 
expert  judgment,  thus  giving  the  PM  the  ability  to  accurately  define  the 
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boundary  location  and  scale  parameters  for  each  cost-overrun  percent¬ 
age  cost  factor— location  giving  the  minimum  bound  and  scale  giving  the 
range  between  maximum  and  minimum  bounds.  This  approach  allows 
the  PM  to  apply  expert  judgment  in  specifying  the  impact  of  cost-risk 
affecting  each  cost  factor.  The  PM  can  easily  modify  cost-overrun  per¬ 
centages  and  fractile  parameters  to  re-calculate  cost-risk  by  introducing 
new  cost-overrun  percentages  and  fractiles  from  the  distribution. 

To  apply  the  probability  model  in  the  decision  analysis,  this  article 
uses  the  Pearson-Tukey  method.  The  Pearson-Tukey  method  gives  the 
PM  the  ability  to  easily  compute  expected  costs  and  their  upper/lower 
bounds  using  the  discrete-form  representation  of  the  beta-general  distri¬ 
bution.  Using  this  technique,  the  division  cost-overrun  percentage  and 
related  probabilities  are  implemented  in  a  three-chance  node  decision 
tree.  The  three-point  approximation  uses  the  .05,  .5,  and  .95  fractiles 
associated  with  the  realization  probabilities  of  18.5  percent,  63  percent, 
and  18.5  percent  respectively.  The  resulting  solution  gives  the  total 
expected  costs  for  the  project,  whereby  each  division  cost  factor  value 
(in  the  tree)  is  accounted  for  in  the  estimating  contractor's  total  cost 
for  the  project.  Thus,  the  DCM  shows  which  contractor  has  the  greatest 
likelihood  of  minimizing  the  total  expected  project  cost  (Clemen,  2001). 

Cost  Factor  Data  Table 

Historic  project  cost  data  are  summarized  in  the  Cost  Factor  Data 
(CFD)  Table.  The  table  contains  a  summary  of  the  contractor's  division 
estimate,  initial  project  cost,  actual  project  cost,  computed  cost-overrun 
percentage  for  each  project,  and  the  Pearson-Tukey  approximated  val¬ 
ues.  Table  1  is  an  example  of  Contractor  1  Mechanical  Cost  Factor.  The 
top  header  contains  the  initial  cost,  actual  costs,  and  cost-overrun  per¬ 
centage.  The  pink  area  contains  the  Mechanical  division  five  previous 
projects,  initial  estimate,  actual  costs,  and  cost-overrun  percentage.  The 
blue  shaded  area  contains  the  preliminary  computed  and  expert  judgment 
applied  minimum,  median,  and  maximum  cost-overrun  percentages.  The 
green  shaded  area  in  Table  1  contains  the  Pearson-Tukey  approximation 
and  the  fractile  cost-overrun  percentage  values  and  probabilities. 
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TABLE  1.  CONTRACTOR  1  MECHANICAL  COST  FACTOR 
DATA  TABLE 


1  Mechanical 

Initial 

Actual 

%  Over 

Renovation  1 

39000 

65000 

0.666666667 

Renovation  2 

78000 

96000 

0.230769231 

Renovation  3 

595000 

635000 

0.067226891 

Renovation  4 

26000 

32000 

0.230769231 

Renovation  5 

464000 

52000 

0.120689655 

Expert  Judgment 

Upper  Bound 

0.666666667 

0.230769231 

Lower  Bound 

0.067226891 

0.067226891 

Median 

0.230769231 

0.175729443 

Pearson-Tu  key 

Value 

Probability 

Fractile 

0.95  0.3839 

0.185 

0.5  0.175 

0.63 

0.05  0.0468 

0.185 

DCM  Methodology 

The  DCM  methodology  may  be  summarized  as  follows:  development 
of  division  cost  factors,  computation  of  division  cost-overrun  percent¬ 
ages,  and  model  of  cost  factors.  There  must  be  a  preliminary  fit  regarding 
a  beta-general  distribution  as  well  as  an  application  of  expert  judgment 
on  a  case-by-case  basis.  Consider  model  minimization  beta  distribution 
from  the  cost  factor,  cost-overrun  percentages,  and  fractiles.  Addition¬ 
ally,  there  should  be  a  utilization  of  the  modeled  output  parameters 
to  generate  a  beta-general  distribution.  The  PM  must  also  apply  the 
Pearson-Tukey  method  to  approximate  the  cost-overrun  percentage  and 
fractile.  The  DCM  methodology  is  described  in  the  following  five-step 
process  to  compute  cost-overrun  percentage  distribution: 

1.  For  each  of  the  three  cost  factors  to  generate  a  preliminary 
beta-general  distribution  from  the  observed  five  cost-overrun 
percentages,  the  PM  uses  expert  judgment,  as  necessary,  to 
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modify  division  cost-overrun  percentage  input  parameters.  The 
PM  will  then  go  to  Step  2  with  a  reasonable  fractile  and  cost- 
overrun  percentage  for  the  cost  factor. 

2.  The  PM  models  the  beta  distribution  parameters.  Many  differ¬ 
ent  software  packages  can  be  used  to  solve  the  formula.  This 
example  uses  an  author-developed  minimization  solver  in  Excel. 
The  PM  observes  the  fitted  distribution  and  determines  if  modi¬ 
fication  to  the  distribution  is  needed.  The  PM  uses  the  computed 
cost-overrun  percentages  and  Comulative  Distribution  Function 
(CDF)  curve  to  estimate  the  fractile  and  bounds  to  create  the 
best  distribution  to  represent  each  cost  factor.  The  model  then 
computes  the  beta  distribution  parameters. 

3 .  The  beta- general  distribution  will  be  generated  by  the  minimiza¬ 
tion  process  as  described  in  Step  2. 

4.  Application  of  the  Pearson-Tukey  method  will  be  used  to  turn 
the  continuous  distribution  into  a  three-outcome  chance  node 
for  the  decision  tree. 

5.  Completion  of  the  decision  tree  will  be  accomplished  in  deter¬ 
mining  which  contractor  has  the  lowest  total  expected  cost  for 
the  project.  The  PM  must  examine  the  input  contractor's  total 
estimate,  cost  factor  division  cost  estimate,  and  the  modeled 
computed  cost-overrun  percentage  as  well  as  the  probability 
parameters,  and  subsequently  figure  them  into  the  DCM  Influ¬ 
ence  Table.  (See  Table  8). 


DCM  Example 

The  beta  distribution  minimization  is  the  key  to  identifying  the  total 
lowest  expected  cost  contractor  for  the  project.  The  DCM  minimization 
distribution  requires  a  fractile,  which  is  determined  to  associate  with 
each  cost-overrun  percentage.  The  modeled  distribution  will  be  demon¬ 
strated  with  real  data  provided  by  the  contractor.  This  creates  a  project 
cost-risk  baseline.  Next,  the  model  distribution  will  be  demonstrated 
with  real  data  and  applied  expert  judgment.  Essentially,  the  applied 
expert  judgment  distribution  utilizes  the  same  division  cost  factor 
fractile  determination  and  cost-overrun  percentage  method.  The  DCM 
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example  is  demonstrated  with  real  cost-overrun  percentage  data  from 
the  Contractor  1  Mechanical  cost  factor.  The  only  difference  with  the 
applied  expert  judgment  example  is  due  to  the  fact  that  the  PM  develops 
a  more  accurate  prediction  of  project  cost-overrun  risk. 

In  general,  fractiles  are  defined  as  the  points  between  the  range  [0, 
1]  in  a  distribution.  The  DCM  may  use  predefined  quartiles  such  as  the 
first  quartile  =  .25,  the  median  =  .5,  and  the  third  quartile  =  .75.  Although 
these  fractiles  may  be  used  for  cost-overrun  percentage  modeling,  a 
more  accurate  fractile  model  is  needed.  Both  examples  demonstrate 
a  more  accurate  method  for  determining  fractiles  for  cost-overrun 
percentages.  The  model  selects  specific  cumulative  probabilities  and 
associates  corresponding  fractiles.  The  cumulative  probabilities  and  the 
fractile  determination  method  used  in  the  model  are  estimated  from  a 
Cumulative  Distribution  Curve  (CDC)  generated  from  fitted  beta  general 
cost-overrun  percentage  distribution.  From  the  CDC,  the  PM  determines 
each  fractile  by  estimating  CDC  distribution  of  the  input  p-values  to  the 
fitted  p-values  fractiles  (Clemen,  2001).  The  key  idea  is  to  determine  the 
best  fractiles  using  fitted  cost-overrun  percentages  that  can  be  applied 
to  the  data  set  for  the  computation  of  the  distribution.  From  the  curve  of 
the  CDC,  the  PM  estimates  the  cumulative  probability  value  of  the  cost- 
overrun  percentage  to  determine  the  CDF  fractile  values  for  the  model. 

For  purposes  of  this  computation,  the  PM  will  associate  each  of 
the  five  cost-overrun  percentage  points  with  five  fractile  values.  Other 
computation  parameters  needed  are  the  extreme  distribution  bounds 
from  the  five  cost-overrun  percentage  data.  The  distribution  of  the  lower 
boundary  will  be  0,  and  the  upper  boundary  will  be  the  cost-overrun  per¬ 
centage  plus  .1.  The  CDC  estimated  the  p-value  to  fitted  p-value  fractile 
is  inputted  into  the  model.  CDF  fractiles  selected  from  the  CDC  are  as 
follows:  point  .1  is  used  as  the  first  fractile,  the  estimated  second  fractile, 
the  median,  a  fourth  fractile,  and  the  fifth  fractile. 

A  general  recommendation  for  fractile  determination  is  to  analyze 
the  computed  median  and  the  upper  bound  P(X  <=  x)  =  .9.  The  assump¬ 
tion  of  the  median  is  the  key  starting  point,  and  the  90  percent  fractile 
is  a  reasonable  upper  boundary  because  it's  a  number  that  construction 
estimators  and/or  construction-contract  managers  can  understand. 
This  model  makes  it  possible  to  determine  the  fractile  values  based  on 
the  CDC  distribution.  The  model  uses  a  three-step  process  to  compute 
five  fractiles  needed  to  define  the  alpha  1,  alpha  2,  and  the  minimum 
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and  maximum  boundaries.  These  parameters  are  needed  to  model  the 
beta-general  distribution  necessary  for  application  of  the  Pearson- 
Tukey  method. 

1.  Step  1:  Fit  the  data. 

2.  Step  2:  Estimate  the  CDC  P-values  to  fitted  p-value  distribu¬ 
tion  points  and  adjust  model  fractile  input  to  compute  fractile. 
Review  match  criteria  of  .1. 

3.  Step  3:  Input  computed  alpha  1,  alpha  2 ,  min  and  max  to  fit  the 
Pearson-Tukey  distribution. 


The  first  step  involves  fitting  the  cost-overrun  Contractor  1  with  the 
Mechanical  cost-overrun  percentage  in  real  data.  Contractor  1  Mechan¬ 
ical  Division  has  five  historical  cost-overrun  percentages  as  follows: 
.0672,  .1206,  .2307,  .2307  and  .6667.  The  data  are  limited  to  five  data 
points  with  a  range  of  6.072  percent  --66.67  percent  cost-overrun  per¬ 
centage.  Note  the  duplicate  23.07  percent  cost-overrun  percentage.  This 
may  be  a  concern,  but  demonstrates  how  the  real  data  are  modeled.  The 
preliminary  beta-general  cost-overrun  data  fit  the  results  in  Figure 
1—28.8  percent  median,  alpha  1  =  .194,  alpha  2  .2307,  min  .067,  and  max 
=  .6667. 
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FIGURE  1.  CONTRACTOR  1  MECHANICAL  COST-OVERRUN 
PERCENTAGE  FITTED  REAL  DATA. 


BetaGeneral  (0.19433, 
0.23070,  0.067226,  0.66667) 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Toots,”  by  R.  T.  Clemen,  2001 


The  second  step  is  to  estimate  the  CDC  p-values  to  fitted  p-value 
distribution  points  and  adjust  model  fractile  input  to  compute  fractile. 
Review  match  criteria  of  .1.  Fit  the  beta-general  distribution,  and  esti¬ 
mate  the  five  fractile  points  using  the  CDC.  Determine  the  fractile  data 
points  by  estimating  each  distribution  point  of  the  p-value  to  the  fitted 
p-value  point  by  evaluation  of  the  slope  of  the  CDC.  Through  evaluation 
of  the  Contractor  1  mechanical  cost-overrun  percentage  real  data  CDC 
curve,  one  can  estimate  the  p-value/fitted  p-value  origin  fractile  as  (0,  .1), 
with  the  first  fractile  at  (.3,  .4),  the  next  fractile  at  (.7,  .5),  and  the  fractile 
termination  at  (.9, 1).  The  points  in  the  parentheses  (x,  y)  are  points  on  the 
CDC  curve.  From  these  estimated  points,  the  first  fractile  is  estimated  .1. 
The  next  fractile,  .2,  is  estimated  from  the  distribution  points  between 
point  (0,  .2)  and  (.3,  .4)  on  the  CDC.  The  next  two  fractiles  are  .49  and 
.5,  and  are  estimated  between  (.3,  .4)  and  (.7,  .5)  on  the  CDC  curve.  The 
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CDC  fractile  termination  will  indicate  the  fifth  fractile  to  be  in  the  90 
percent  quartile.  Using  this  estimation  method,  the  PM  can  estimate  the 
CDF  fractile  model  values  as  shown  in  figure  2  (Clemen,  2001,  p.  403). 

FIGURE  2.  CONTRACTOR  1  MECHANICAL  REAL  DATA  FITTED 
P-VALUE/INPUT  P-VALUE  CURVE 


BetaGeneral  (0.19433, 
0.23070,  0.067226,  0.66667) 


Input  p-value 


Note.  Adapted  from  ‘‘Making  Hard  Decisions  with  Decision  Tools,"  by  R.  T.  Clemen,  2001. 

The  PM  will  model  the  minimization  beta  distribution.  The  PM  will 
input  each  cost-overrun  percentage  in  progression  with  the  associated 
fractile  into  the  model;  6.72  percent  =  .1, 12.06  percent  =  .2, 23.07  percent 
=  .49, 23.07  percent  =.5,  and  66.67  percent  =  .9.  The  model  utilizes  a  beta 
distribution,  which  returns  the  cumulative  beta  probability  density  of 
the  inputted  cost-overrun  percentage  and  the  inputted  CDF  fractile. 
The  result  is  a  computed  fractile,  which  the  PM  compares  to  the  inputted 
CDF  fractile.  If  the  computed  fractile  is  within  .01  of  the  inputted  CDF 
fractile,  the  PM  considers  this  a  computed  fractile  match.  The  PM  then 
utilizes  the  modeled  output  parameters— alpha  1,  alpha  2,  and  the  min 
and  max— to  fit  the  beta-general  distribution. 
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The  Contractor  1  Mechanical  cost-overrun  real  example  results  are 
alpha  1  =  1.55,  alpha  2  =  3.026,  min  =  0,  and  max  =  .73.  The  parameters  are 
used  to  fit  a  beta-general  distribution  for  estimation  of  the  Pearson-Tukey 
overrun  percentage  and  probability  values  used  in  the  decision  tree.  Table 
2  displays  the  model  with  the  beta-distribution  computation  of  the  fractile 
from  the  cost-overrun  percentage  and  CDF  fractile  inputs.  Note  the  CDF 
fractile  is  computed  within  the  .15  range— a  PM-considered  match.  This 
will  indicate  that  the  alpha  1,  alpha  2,  and  the  min  and  max  are  ready  for 
the  next  step— fit  the  beta- general  Pearson-Tukey  distribution. 

TABLE  2.  CONTRACTOR  1  MECHANICAL  FRACTILE  REAL 
DATA  SOLVER 


Step  2: 

Objective 

Function 

0.133445735 

alpha  1 

1.550436497 

alpha  2 

3.026241986 

max 

0.735866163 

min 

0 

0.248449 

Overrun  % 

CDF  Fractile 

Computed 

0.0672269 

0.1 

0.1 

Fitted 

0.1206897 

0.2 

0.225208 

0.249222 

0.230769 

0.49 

0.499879 

0.230769 

0.5 

0.499879 

0.66667 

0.9 

0.998237 

The  third  step  involves  determining  the  Pearson  Tukey  overrun 
percentage  and  probability  values.  The  computed  model  results  are  as 
follows:  alpha  1  =  1.55,  alpha  2  =  3.02,  min  =  0,  and  max  =  .73.  Parameters 
are  used  to  model  the  beta-general  distribution.  The  Pearson-Tukey 
method  is  applied  to  identify  the  cost-overrun  percentages  and  5  percent, 
median,  and  95  percent  probabilities  for  the  decision  tree.  Through  appli¬ 
cation  of  the  Pearson-Tukey  method,  the  PM  estimates  the  95  percent 
fractile  is  equal  to  the  probability  of  18.5  percent,  with  a  cost-overrun 
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of  52  percent.  The  median  fractile  probability  is  equal  to  63  percent, 
with  a  23.07  percent  cost-overrun.  The  5  percent  fractile  is  equal  to  the 
probability  of  18.5  percent,  with  a  cost-overrun  of  4.18  percent.  These 
values  are  entered  into  the  decision  tree  to  compare  each  contractor's 
project  cost-overrun  risk  to  their  project  completion.  Figure  3  depicts 
the  Contractor  1  Mechanical  real  data  fitted  beta-general  distribution 
results  from  the  modeled  parameters. 

FIGURE  3.  CONTRACTOR  1  MECHANICAL  REAL  DATA 
DISTRIBUTION 

BetaGeneral  (1.550,  3.0262,  0,  .7358) 

3.0 

2.5 

2.0 

1.5 

1.0 

0.5 

0.0 

,-:q,-:rMhOTrLntoivco 

?d°odddddd 


90.0% 


5.0% 


0.0418 


0.5200 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Tools/’  by  R.  T.  Clemen ,  2001 

Fractile  Determination  with  Applied 
Expert  Judgment 

Figure  4  reflects  the  Contractor  1  Mechanical  cost-overrun  per¬ 
centage  with  applied  expert  judgment.  The  modeling  and  fractile 
determination  process  are  the  same  as  modeling  with  real  data.  The 
main  difference  is  the  PM  applies  expert  judgment  to  the  real  data  to 
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adjust  cost-overrun  outliers.  Because  of  limited  data  provided  by  the 
contractor,  the  PM  must  rely  on  experience  and  sound  judgment  to 
make  any  adjustments  to  the  data  set.  Using  expert  judgment,  the  PM 
modifies  distribution  parameters  to  best  represent  the  contractor.  The 
PM  accomplishes  the  modification  by  careful  evaluation  of  the  project 
scope  of  work  to  be  performed,  division  cost-overrun  percentage  range, 
and  identification  of  cost-overrun  outliers. 

The  PM  must  also  understand  the  many  factors  that  impact  a  cost- 
overrun  risk  on  a  construction  project.  The  PM  uses  past  experience 
to  reasonably  evaluate  the  cost-overrun  risk  to  the  project.  Some  com¬ 
mon  cost-overrun  examples  are:  unclear  documented  scope  of  work, 
unforeseen  problems,  project  location,  and  abatement  of  facility.  These 
examples  are  common  and  add  enormous  cost  to  a  construction  project. 
A  prime  contractor's  project  experience  on  special  projects  and  a  prime 
contractor's  experience  with  the  subcontractor  also  impact  the  cost- 
overrun  risk.  Lower  cost-overrun  risk  occurs  when  a  prime  contractor 
has  an  established,  longstanding  relationship  with  a  subcontractor.  The 
project  tends  to  run  more  effectively  with  better  cost  control. 

The  construction  industry  identifies  the  contractor  responsible  for 
the  overall  project  as  the  “prime  contractor.''  The  subcontractor  works 
for  the  “prime  contractor."  An  example  of  a  prime  contractor  with  a  lim¬ 
ited  working  relationship  with  a  subcontractor  is  what  the  construction 
industry  calls  a  construction  broker.  These  construction  brokers  esti¬ 
mate  a  construction  project  and  hire  local  subcontractors  to  complete  the 
project.  Because  of  lower  overhead  and  remote  capability,  a  construction 
broker's  estimate  may  be  lower  than  other  prime  contractors.  Because 
of  limited  working  relationships  with  local  subcontractors,  the  prime 
contractor  broker  incurs  large  project  cost-overruns.  Other  cost-overrun 
examples  include:  subcontractor  experience,  project  location  (whether 
the  project  is  in  a  city  or  in  the  middle  of  a  desert),  and  weather  (such 
as  snow,  wind,  and  rain).  All  these  variables  influence  the  PM's  expert 
judgment  application  to  division  cost  factors. 

Every  division  cost  factor  is  modeled  independently  of  one  another. 
For  example,  the  mechanical  cost-overrun  percentage  does  not  depend 
on  the  cost-overrun  (or  any  other)  estimate  such  as  electrical  or  finish. 
Division  cost  factors  such  as  mechanical  and  electrical  are  primarily 
managed  by  independent  subcontractors.  The  finish  division  cost  factor 
is  also  independent  from  the  other  cost  factors  and  is  primarily  managed 
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by  the  “prime”  contractor.  The  prime  contractor  is  the  contractor  who 
is  responsible  overall  for  the  project  and  the  subcontractors'  division 
impacts. 

The  DCM  process  is  accomplished  in  the  following  Steps  1  through  3. 

Step  1 

The  first  step  is  to  review  the  divisional  “real”  data  and  apply  expert 
judgment  to  determine  the  most  likely  cost-overrun  percentage  and,  if 
needed,  determine  the  least  likely  cost-overrun  percentage.  The  obser¬ 
vation  of  Contractor  1  Mechanical  cost-overrun  percentage  is  that  4  of 
the  5  cost-overrun  percentages  are  in  the  range  between  .067-. 2307.  The 
PM  has  determined  that  the  .6667  cost  overrun  appears  to  be  an  outlier. 
This  cost-overrun  is  from  a  lower  cost  mechanical  project  where  minor 
changes  in  cost  amplify  a  larger  change  in  cost-overrun  percentage.  This 
mechanical  project  initial  estimate  was  $39,000,  and  the  final  actual  cost 
was  $65,000.  After  review  of  the  contractor's  initial  proposal,  the  PM  had 
determined  the  contractor  initially  estimated  this  mechanical  division 
estimate  as  a  repair  of  the  existing  mechanical  system.  The  contractor's 
good-faith  estimate  was  proposed  to  save  materials,  labor,  and  the  ability 
to  use  the  existing  system.  After  further  analysis,  the  mechanical  project 
became  a  total  mechanical  replacement,  thus  reflected  in  the  actual  cost, 
not  in  the  good-faith  estimate. 

With  limited  data  to  evaluate  the  contractor,  and  judging  from  the 
cost-overrun  percentages,  4  of  the  5  are  less  or  equal  to  23.07  percent 
cost-overrun  percentage.  More  than  likely,  a  23.07  percent  or  smaller 
cost-overrun  may  occur,  while  a  cost-overrun  of  66.67  percent  is  least 
likely.  Contractor  1  proposed  this  mechanical  project  for  $480,000.  From 
the  contractor  project  real  data,  the  contractor's  two  similar  mechanical 
projects  were  for  $595,000  and  $464,000;  the  contractor  had  a  6.7  per¬ 
cent  and  12  percent  cost-overrun  respectively.  Because  of  risk  to  costs, 
the  PM  cannot  completely  discount  the  66.67  percent  cost-overrun,  so 
the  PM  will  use  the  complete  contractor  mechanical  real  cost-overrun 
dataset  (.067-. 66667)  to  compute  the  median  .2881.  The  high  median 
is  driven  by  the  one  high  .6667  cost-overrun  percentage.  The  PM  will 
replace  the  .6667  cost-overrun  percentage  with  the  median  .2881  and 
input  into  the  cost-overrun  percentage  in  the  5th  fractile  of  the  model. 
Keep  in  mind  that  this  will  be  the  only  applied  expert  judgment  made  to 
the  Contractor  1  Mechanical  overrun  percentage  dataset. 
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The  Contractor  1  Mechanical  real  cost-overrun  percentage  (Figure 
1)  fits  beta-general  distribution  with  a  computed  median  .2881.  The  PM 
determines  this  is  abetter  representation  of  the  Contractor  1  Mechanical 
cost-overrun.  Next,  determine  the  fractiles  using  the  developed  model 
and  proceed  to  Step  2. 

FIGURE  4.  CONTRACTOR  1  MECHANICAL  APPLIED  EXPERT 
JUDGMENT  REAL  DISTRIBUTION 


BetaGeneral  (0.22171, 
0.21318,  0.067227,  0.28810) 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Tools,”  by  R.  T.  Clemen,  2001 


Step  2 

Estimate  the  CDC  P-values  to  fitted  p-value  distribution  points 
and  adjust  the  CDF  fractile  input  to  compute  the  model  fractile.  In  this 
example,  the  fractile  determination,  the  PM  demonstrates  Contractor 
1  Mechanical  real  data  with  applied  expert  judgment.  From  the  curve 
estimate,  the  p-value/ fitted  p-value  origin  fractile  is  (0,  .1),  the  second 
fractile  point  is  (.3,  .4),  the  third  fractile  at  (.5,  .5),  the  fourth  fractile  is 
(.7,  .6),  and  the  fractile  termination  is  (.9, 1).  From  the  CDC  estimated 
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Fitted  p-value 


points,  the  model  fractiles  are  estimated  to  be  .1,  .3,  .55,  and  .75.  The  CDC 
.9  fractile  termination  is  the  model  fifth  fractile.  The  fractile  estimated 
values  are  inputted  into  the  CDF  fractile  model. 


FIGURE  5.  CONTRACTOR  1  MECHANICAL  EXPERT  APPLIED  DATA 
P-VALUE  CDF 

BetaGeneral  (0.22171,  0.21318,  0.067227,  0.28810) 


Input  p-value 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Tools/’  by  R.  T.  Clemen ,  2001 

Observations  on  Table  3: 

1.  Alpha  1  =  2.596,  alpha  2  =  6.303,  min  =  0,  max  =  .585. 

2.  Application  of  PM  expert  judgment,  the  .6667  cost-overrun  is 
replaced  with  .2881  median. 

3.  The  lstfractile  .1,  .3,  .75,  and  .9  is  within  the  .15  match  criterion. 
The  3rd  fractile,  .2307,  is  a  duplicate  of  the  4th  fractile,  which 
accounts  for  the  .764  computed  median  fractile.  The  .184  fitted 
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median  and  .2881  upper  bound  model  the  applied  expert  judg¬ 
ment  shape  or  bounds  for  the  distribution.  The  PM  uses  the 
modeled  parameters  to  fit  the  beta-general  distribution. 

TABLE  3.  CONTRACTOR  1  MECHANICAL  EXPERT  JUDGMENT 
APPLIED  DATA  SOLVER 


Step  2: 

Expert  judgment  decision  to  discard  upper  bound  and  insert  real 

data  fitted  median  .2881 

Objective 

Function 

0.243971668 

alpha  1 

2.596224508 

alpha  2 

6.303480736 

max 

0.58571502 

min 

0 

Overrun  % 

CDF  Fractile 

Computed 

Fitted  Dist 

0.0672269 

0.1 

0.1 

0.179829619 

0.1206897 

0.3 

0.314529 

0.2307692 

0.55 

0.764705 

0.2307692 

0.75 

0.764705 

.6667  replaced 

with  fitted  median 

0.2881 

0.9 

0.899968 

Step  3 

Determine  the  Pearson  Tukey  values.  The  model  parameter  results 
are  alpha  1  =  2.5906,  alpha  2  =  6.303,  minimum  =  0,  and  maximum  =  .585 
are  used  to  fit  the  beta-general  distribution.  The  Pearson-Tukey  method 
is  applied  to  approximate  the  median,  upper,  and  lower  cost-over-per¬ 
centage  and  probabilities  for  the  decision  tree.  The  Pearson-Tukey 
method  estimates  the  95  percent  fractile  is  equal  to  the  probability  of  18.5 
percent,  with  a  cost-overrun  percentage  of 32.46  percent.  The  median  is 
equal  to  63  percent,  with  a  16  percent  cost-overrun.  The  lower  5  percent 
fractile  is  equal  to  the  probability  of  18.5  percent,  with  a  cost-overrun 
percentage  of  4.88  percent.  These  values  are  entered  into  the  decision 
tree  to  compare  each  contractor  project  cost-overrun  risk  to  their  project 
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estimate.  In  Figure  6,  the  Pearson-Tukey  cost-overrun  percentage  and 
probability  values  are  entered  into  the  decision  tree  to  compare  each 
contractor  project  cost-overrun  risk  to  their  project  completion. 

FIGURE  6.  CONTRACTOR  1  MECHANICAL  PEARSON-TUKEY  WITH 
EXPERT  JUDGMENT  APPLIED  DATA 

BetaGeneral  (2.59,  6.303,  0,  .585) 

5.0 

4.5 
4.0 

3.5 
3.0 

2.5 

2.0 

1.5 
1.0 
0.5 
0.0 

TO^cMro^-inio 

?d°odddd 


90.0% 


5.0% 


0.0488 


0.3246 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Tools/’  by  R.  T.  Clemen ,  2001 

In  summation,  the  PM  first  demonstrated  the  model  with  limited  real 
data  provided  by  the  contractor.  The  model  was  then  demonstrated  with 
the  application  of  expert  judgment  to  the  same  Contractor  1  Mechani¬ 
cal  real  data.  The  DCM  results  show  that  with  the  provided  real  data, 
and  without  the  application  of  expert  judgment,  Contractor  1  would  be 
the  suggested  contractor  for  project  award  with  a  total  project  cost  of 
$5,571,137.  The  DCM  results,  with  applied  expert  judgment  to  the  real 
data,  demonstrate  that  Contractor  2  is  the  suggested  contractor  for  proj¬ 
ect  award,  with  a  total  project  cost  of  $5,438,781. 
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DCM  Data  Summary 

The  DCM  model  input  and  output  parameters  are  in  Tables  4-7. 
Table  4  contains  the  model  parameters  for  each  contractor's  real  data 
cost  factor  distribution  shape  parameters.  Table  5  contains  the  contrac¬ 
tor's  real  data  cost  factor  Pearson-Tukey  cost-overrun  percentage  and 
probability  values.  Table  6  contains  the  model  cost  factor  distribution 
parameters  for  each  contractor's  real  data  with  applied  expert  judg¬ 
ment.  Table  7  contains  the  summarized  Pearson-Tukey  cost-overrun 
percentage  and  probability  values  for  contractor's  real  data  cost  factor 
with  the  applied  expert  judgment. 

Table  4  is  the  summary  of  the  output  model  parameters  for  each 
contractor.  The  model  parameters  computed  with  real  data  provided  by 
the  contractors.  The  computed  parameters  did  not  have  any  expert  judg¬ 
ment  applied.  The  output  parameters  are  used  to  define  each  contractor's 
division  cost  factor  distribution  shape. 

TABLE  4.  CONTRACTOR’S  DIVISION  COST  OVERRUN  MODEL 

OUTPUT  PARAMETER  RESULTS  (NO  EXPERT  JUDGMENT  APPLIED) 


Contractor  1 

*1 

Min 

Max 

Mechanical 

1.550 

3.026 

0 

.735 

Finish 

.562 

4.371 

0 

.313 

Electrical 

2.252 

14.056 

0 

.487 

Contractor  2 

*i 

oC2 

Min 

Max 

Mechanical 

1.354 

2.665 

0 

.850 

Finish 

2.269 

3.88 

0 

.254 

Electrical 

.968 

.959 

0 

.628 

Contractor  3 

*i 

oC2 

Min 

Max 

Mechanical 

1.052 

.976 

0 

.72 

Finish 

2.382 

5.383 

0 

.914 

Electrical 

.641 

.698 

0 

.709 

Table  5  is  a  summary  of  the  Pearson-Tukey  cost-overrun  percentage 
and  probability  for  each  contractor's  real  data  division  cost  factor.  The 
Pearson-Tukey  cost-overrun  percentage  and  probability  approximation 
will  be  inputted  to  create  the  decision  tree. 
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TABLE  5.  CONTRACTOR’S  DIVISION  PEARSON-TUKEY  REAL  DATA 
COST-OVERRUN  RESULTS  (NO  EXPERT  JUDGMENT  APPLIED) 


Contractor  1 

18.5%  probability 

63%  probability 

18.5%  probability 

Mechanical 

4.18% 

23.07% 

52% 

Finish 

.08% 

5.21% 

313% 

Electrical 

1.481% 

5.931% 

14.2% 

Contractor  2 

18.5%  probability 

63%  probability 

18.5%  probability 

Mechanical 

3.9% 

26.3% 

61% 

Finish 

3.03% 

10.5% 

19.4% 

Electrical 

2.88% 

31.2% 

59.2% 

Contractor  3 

18.5%  probability 

63%  probability 

18.5%  probability 

Mechanical 

4.2% 

37.8% 

72.0% 

Finish 

7.5% 

26.4% 

54% 

Electrical 

1% 

32.88% 

68.48% 

Table  6  is  the  summary  of  the  output  model  parameters  for  each 
contractor.  The  model  parameters  were  computed  with  the  application 
of  expert  judgment  to  the  real  data.  These  parameters  are  used  to  define 
the  cost  factor  distribution  shape. 

TABLE  6.  CONTRACTOR’S  DIVISION  COST-OVERRUN  MODEL  OUTPUT 
PARAMETERS  RESULTS  (WITH  EXPERT  JUDGMENT  APPLIED) 


Contractor  1 

*1 

«2 

Min 

Max 

Mechanical 

1.948 

2.024 

0 

.359 

Finish 

2.246 

3.144 

0 

.571 

Electrical 

3.006 

10.283 

0 

.2307 

Contractor  2 

*2 

Min 

Max 

Mechanical 

2.262 

8.266 

0 

.72 

Finish 

2.004 

4.063 

0 

.439 

Electrical 

5.086 

7.567 

0 

.261 

Contractor  3 

ccz 

Min 

Max 

Mechanical 

2.216 

2.146 

0 

.388 

Finish 

1.937 

2.173 

0 

.600 

Electrical 

.641 

.698 

0 

.709 
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Table  7  contains  the  summary  of  the  Pearson-Tukey  cost-overrun 
percentage  and  probability  for  each  contractor's  division  cost  factor  real 
data  with  applied  expert  judgment.  The  Pearson-Tukey  cost-overrun 
percentage  and  probability  approximation  are  used  to  create  the  deci¬ 
sion  tree. 

TABLE  7.  CONTRACTOR’S  DIVISION  PEARSON-TUKEY  COST- 
OVERRUN  PERCENTAGE  RESULTS  REAL  DATA  WITH  APPLIED 
EXPERT  JUDGMENT. 


Contractor  1 

18.5%  probability 

63%  probability 

18.5%  probability 

Mechanical 

4.48% 

17.1% 

30.06% 

Finish 

6.05% 

23.1% 

43.1% 

Electrical 

1.61% 

4.87% 

9.87% 

Contractor  2 

18.5%  probability 

63%  probability 

18.5%  probability 

Mechanical 

4.86% 

16.2% 

33.91% 

Finish 

3.26% 

13.35% 

20.86% 

Electrical 

5.1% 

10.3% 

16.37% 

Contractor  3 

18.5%  probability 

63%  probability 

18.5%  probability 

Mechanical 

5.94% 

19.8% 

33.39% 

Finish 

7.18% 

27.9% 

50.41% 

Electrical 

1.01% 

33.08% 

69.37% 

Table  8  contains  the  Decision  Cost  Model  influence  summary  with 
the  application  of  expert  judgment  to  the  real  data  provided  by  the  con¬ 
tractor.  The  DCM  Influence  Summary  input  parameters  are  initial 
estimate,  division  cost  factor  estimate,  and  computed  Pearson-Tukey 
cost-overrun  percentage  and  probability.  The  DCM  computed  the  model 
and  identified  that  Contractor  2  had  the  lowest  variability  of  division  cost 
overruns,  resulting  in  the  selection  of  Contractor  2  as  the  “best  value” 
contractor,  with  a  project  estimated  expected  total  cost  of  $5,438,781. 
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TABLE  8.  DECISION  COST  MODEL  SUMMARY  (WITH  APPLIED  EXPERT  JUDGMENT) 


335 


Defense  ARJ,  July  2012,  Vol.  19  No.  3 : 309-344 


Decision  Cost  Model  for  Contractor  Selection 


Table  9  contains  the  Statistics  and  Risk  Profile  charts  generated 
from  the  model.  The  charts  provide  an  analytical  data  summary  to  the 
end  user  for  this  decision.  The  statistics  chart  shows  the  range  for  each 
contractor's  estimate  with  the  cost  factor  inputs.  The  model  identifies 
Contractor  3,  who  initially  had  the  lowest  project  estimate,  as  the  con¬ 
tractor  with  the  highest  total  expected  cost  of  the  three  contractors. 
Contractor  3  has  a  cost  range  from  $5,111,140  to  $6,387,421,  with  a  mean 
cost  of  $5,742,003.  Contractor  1,  who  had  the  highest  initial  estimate,  is 
the  second  lowest  total  expected  cost  contractor.  Contractor  1  has  costs 
that  range  from  $5,461,365  to  $6,082,365,  with  a  mean  cost  of  $5,720,684. 
Contractor  2,  the  model-recommended  contractor,  had  costs  that  ranged 
from  $5,129,768  to  $5,687,267,  with  the  mean  cost  of  $5,438,781. 

Table  9  also  displays  total  cost  variability  by  contractor.  Contractor 

1  has  the  cost  standard  deviation  of  $138,554,  and  Contractor  3  has  the 
highest  variability  of  cost  standard  deviation  of  $234,952.  Contractor  2 
has  the  lowest  cost  standard  deviation  of  $88,110.  The  risk  profile  chart 
in  Table  9  displays  how  each  contractor's  cost  probability  and  overruns 
are  distributed.  This  gives  the  decision  maker  confidence  for  the  decision 
and  provides  further  support  for  the  selection  of  Contractor  2. 

The  total  expected  cost  risk  profile  for  the  project  demonstrates  that 
Contractor  1  has  a  35  percent  confidence  the  contractor  will  meet  the 
computed  “mean”  total  expected  cost  of  the  project,  and  that  Contractors 

2  and  3  both  have  a  30  percent  confidence  they  will  meet  the  computed 
“mean”  total  expected  cost  of  the  project.  The  cumulative  probability 
plot  shows  that  Contractor  1  and  Contractor  3  are  grouped  together  with 
total  expected  cost  risk.  Contractor  2  has  separated  from  the  other  two 
contractors'  project  selections  and  offers  the  lowest  total  expected  cost 
with  the  highest  confidence. 
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TABLE  9.  CONTRACTOR’S  STATISTICS,  RISK  AND  CUMULATIVE 
PROBABILITY  PROFILE 


Contractor 

Selection 

7;  Contractor  7 

2:  Contractor  2 

J;  Contractor  3 

STATISTICS  ! 

Mean 

-5720684 

-5438781 

-5742003 

Minimum 

-6082365 

-5687267 

-6387421 

Maximum 

-5461365 

-5212076 

-5129578 

Mode 

-5667425 

-5432385 

-5732314 

Std  Dev 

138554.1 

88110.91 

234952 

Skewness 

-0.29076 

-0.25133 

-0.10148 

Kurtosis 

2.806697 

2.920348 

2.873793 

Risk  Profile  For  Converted  Diagram  #1  of 
MEMexpertjudgementmodellll.xIsx 


■  1:  Contractor  1 

■  2:Contractor  2 

■  3:  Contractor  3 


Cumulative  Probability  For  Converted  Diagram  #1  of 


1:  Contractor  1 
2:Contractor  2 
3:  Contractor  3 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Tools/’  by  R.  T.  Clemen ,  2001. 
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Figure  7  demonstrates  the  DCM  minimum  expected  cost  solution 
decision  tree.  The  decision  tree  will  demonstrate  how  each  division 
estimate  is  impacted  by  the  computed  division  cost-overrun  percentage 
and  probability.  The  DCM  utilized  the  top  three  divisions  determined 
by  the  PM  as  cost  factors  for  the  decision  tree.  The  DCM  demonstrates 
how  the  mechanical,  electrical,  and  finish  division  cost  factors  have  an 
overall  impact  on  the  total  cost  for  the  project.  The  DCM  will  also  dem¬ 
onstrate  how  the  initial  estimate  from  the  contractor  is  not  the  expected 
cost  provided  to  the  government.  To  better  represent  the  decision  tree 
in  this  report,  the  single  decision  tree  is  shown  in  Figure  7  as  Decision 
Tree  Contractor  Selections,  Parts  1  and  2.  The  decision  tree  chance  node 
demonstrates  how  each  Pearson-Tukey  chance  and  percentage  impact 
each  division  cost  for  the  project,  and  collectively  impact  total  cost  of 
the  project. 


FIGURE  7.DECISION  TREE  CONTRACTOR  SELECTIONS 
PARTI 


■  Contractor  Selection 
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FIGURE  7.DECISION  TREE  CONTRACTOR  SELECTIONS 
PART  2 


Note.  Adapted  from  “Making  Hard  Decisions  with  Decision  Tools/’  by  R.  T.  Clemen ,  2001 
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CONCLUSIONS 

In  summation,  from  a  pool  of  certified  contractors,  the  government 
contracting  office  solicited  estimates  for  the  project.  Utilizing  RSMeans 
as  a  standard  format  for  construction  costs,  the  PM  completed  a  current 
year  IGE,  collected  five  similar  project  estimates,  and  final  project  costs 
from  the  three  potential  contractors.  After  review  of  each  contractor's 
project  history  and  computing  the  division  cost- overrun  percentage,  the 
PM  identified  three  common  cost  factors  for  the  DCM. 

The  PM  fit  a  primary  distribution  for  each  of  the  three  division  cost- 
overrun  percentages.  With  the  historical  project  dataset  and  expert 
judgment,  the  PM  modeled  a  minimization  beta  distribution  to  best 
represent  each  contractor's  cost  factor.  The  model  computed  output 
parameters  used  to  generate  a  beta-general  distribution.  The  PM  applied 
the  Pearson-Tukey  method  to  approximate  the  cost-overrun  percentage 
and  probability  for  each  cost  factor.  The  modeled  cost-overrun  percent¬ 
age  and  probabilities  are  imputed  into  the  DCM  Influence  Table.  With 
the  modeled  cost  overrun  percentages  and  fractiles,  the  total  estimate, 
and  cost  factor  division  estimates,  the  DCM  computed  the  lowest  total 
expected  cost  contractor  for  the  construction  project. 

Initially,  each  contractor  presented  a  total  cost  estimate  for  the 
construction  project.  Contractor  l's  estimate  was  $5.3  million,  Contrac¬ 
tor  2's  estimate  was  $5.1  million,  and  Contractor  3's  estimate  was  $4.9 
million.  Contractor  3  appears  to  be  the  lowest  total  cost  contractor  for 
the  project.  With  the  current  contractor  selection  process,  Contractor  3 
would  have  been  awarded  the  construction  project.  With  the  same  data 
from  the  contractor's  initial  cost  estimate,  cost  factor  division  cost  esti¬ 
mates,  modeled  cost-overrun  percentages,  and  chance  parameters,  the 
DCM  model  demonstrated  that  a  lower  total  expected  cost  decision  for 
the  construction  project  maybe  made.  The  DCM  provides  a  valid,  data- 
driven  decision  process  to  select  the  contractor  best  suited  to  meet  the 
tax-payers'  objective— a  value-driven  government  construction  project. 

Future  application  of  the  DCM  is  a  software  program  that  can  be 
developed  and  added  to  RSMeans  CostWorks  to  streamline  the  contrac¬ 
tor  evaluation  process.  The  DCM  is  not  limited  to  construction  projects. 
The  DCM  can  be  adapted  to  any  problem  with  defined  variables  and 
historical  costs.  The  decision  model  can  be  used  for  private,  municipal, 
state,  and  federal  construction  projects. 
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Annually,  many  U.S.  Government  construction  projects  and  funds 
need  to  be  obligated  for  projects.  The  government  PM,  at  times  will  look 
at  a  contractor's  project  estimates  at  face  value.  A  common  scenario  in 
the  government  construction  process  is  that  a  government-certified 
construction  contractor  will  state,  “It  will  cost  $5  million  dollars  to 
construct  a  facility."  If  the  government  has  the  facilities  project  pro¬ 
grammed,  and  the  IGE  is  within  25  percent  of  the  contractor's  estimate, 
the  government  will  obligate  the  funds  to  the  construction  project.  The 
project  will  be  funded  without  the  knowledge  of  the  contractor's  project 
cost-overrun  percentage  history  and  the  potential  unknowns  surround¬ 
ing  project  cost  overrun. 

Under  the  current  government  contractor  selection  process,  the 
government  would  have  awarded  the  project  to  Contractor  3,  who  had 
the  lowest  initial  estimate  of  $4.9  million.  The  DCM  demonstrated  that 
Contractor  3  is  not  the  lowest  cost,  but  indeed  has  the  largest  cost  risk 
for  project  award. 

The  author's  experience  in  construction  management  and  evaluation 
of  project  historical  cost  data  indicates  the  majority  of  construction  proj¬ 
ects  will  have  at  least  a  division  cost-overrun.  Cost-overruns  are  often 
termed  by  the  contractor  as  “modification,  change  order,  or  upgrade."  On 
several  occasions,  a  contractor  proposes  to  win  a  government  project  by 
bidding  the  lowest  estimate.  The  contractor  later  makes  up  the  differ¬ 
ence  in  modifications  or  change  orders  throughout  the  project,  as  was 
demonstrated  by  DCM  in  Contractor  3's  situation. 

The  recommendation  of  this  study  is  for  U.S.,  state,  and  municipal 
governments  to  take  careful  consideration  of  construction  division  cost- 
overruns  before  project  contractor  project  award  selection.  This  article 
demonstrated  that  by  utilizing  a  good  DCM  and  a  common  format,  a 
valid,  data-driven  decision  can  be  made  for  project  award.  Using  this 
process  will  bring  more  cost-effective  contractor  selection  solutions 
for  the  government  and  construction  engineers.  Using  this  DCM,  the 
federal  government's  stimulus  and  project  funding  could  be  used  more 
efficiently,  thus  meeting  the  taxpayers'  expectations  of  responsible  gov¬ 
ernment  construction  spending  for  their  tax  dollars. 
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Review: 

Alfred  Chandler's  The  Visible  Hand  contributes  significant  insights  into 
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managerial  capabilities.  This  has  important  implications  in  understanding 
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spectives  in  the  book  can  help  in  assessing  whether  key  functions  should  be 
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of  these  functions  in  smaller,  diverse  companies  led  to  reduced  transaction 
costs  in  conducting  core  functions,  as  well  as  greater  productivity.  Chandler 
focuses  on  the  importance  of  the  creation  of  managerial  hierarchies  as  well  as 
the  development  of  the  formal  profession  of  managers  in  achieving  the  inter¬ 
nalization  of  activities  and  in  the  formation  of  large,  American  companies. 
Without  the  development  of  professional  managers,  the  benefits  of  improved 
productivity  and  lower  costs  due  to  the  synergies  between  the  functional 
units  and  their  integration  into  the  broader  corporate  structure  could  not 
have  been  realized. 

Chandler  examines  the  managerial  revolution  in  a  variety  of  industries, 
including  the  evolution  of  the  railroad  industry  in  the  United  States.  The 
analysis  examines  changes  in  mass  distribution  (the  development  of  depart¬ 
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through  mergers  in  a  variety  of  industries.  These  historical  examples  have 
many  parallels  with  contemporary  supply  chain  management  challenges, 
some  of  which  have  been  effectively  dealt  with  through  acquisition  of  smaller 
companies  conducting  core  functions  into  larger  corporate  enterprises. 

An  understanding  of  how  the  “visible  hand”  of  management  replaced  the 
“invisible  hand”  of  market  mechanisms  through  the  evolution  of  the  modern 
American  business  enterprise  and  through  the  associated  development  of 
managerial  hierarchies  is  key  in  evaluating  the  challenges  currently  facing 
American  industries,  including  the  defense  sector.  When  I  was  a  PhD  student 
at  Harvard  University,  I  always  found  that  Alfred  Chandler's  perspectives  in 
his  discussions  with  the  students  in  class  provided  valuable  insights  on  how 
historic  and  contemporary  threads  were  woven  together  to  create  the  tapes¬ 
tries  of  particular  industries  or  markets.  Today,  both  the  economic  crisis  and 
the  associated  budgetary  pressures  necessitate  improved  efficiencies,  greater 
productivity,  and  reduced  costs  in  the  industrial  base,  and  the  exploration  of 
economic  history  in  The  Visible  Hand  can  provide  the  foundations  for  some 
possible  solutions. 
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provide  us  with  your  business  e-mail  address,  you  may  become  part  of  a  mailing 
list  we  are  required  to  provide  to  other  agencies  who  request  the  lists  as  public 
information.  If  you  prefer  not  to  be  part  of  these  lists,  please  use  your  personal 
e-mail  address. 


ver  11/28/11 
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